An interesting thread problem

Interesting problem

I am trying to flesh out the thread locking for our embedded python interpreter. A user can write scripts for our application, and if a script defines a callback it can respond to events from the application (like a midi note or musical clock tick).

As it stands, we are not using process migration to allow for true multiprocessing, but instead are just hamming a single lock from multiple threads to protect the CPython library. This is good enough for now - it works quite well at relatively low latencies.

I compiled python without thread management to avoid an incredible number of 'NULL tstate' errors caused by unpredictable thread management by sequencing host applications. The app is solid, but now I want to integrate a simple stack frame mechanism to allow scripts to behave a little better when calling other scripts. Short intro, but the details don't matter anyway...

Here's the deal

I have one audio engine per thread, and several scripts per engine. The currently executing script is stored as a pointer on the engine object using my function setActiveScript(), and when a new script is called it is swapped with the current one, much like PyThreadState_Swap(). This in effect creates a stack of scripts in the engine, where when execution returns to the original script NULL is passed to setActiveScript().

This works great, but when there is more than one engine (and so more than one thread), I need to acquire a critical section lock and swap the engine for another one, while maintaining the current script on that engine. The script pointer is stored in the engine, but where to enter and exit the critical section is tough. Hairy.

The function looks like this:

Script *ScriptingApplication::instance()->setActiveScript(Script *)

I haven't written much complex code in the last while, so lining out the algorithm and thread locks is kind of clunky at best. it doesn't help that Ableton Live (a host that loads our audio plugin) suddenly doesn't want to be debugged.

*crash*. uggh.

Pure Art

There comes a point when a piece of art is so refined that the products, byproducts, and even messiest and most defiled anty-products of that piece of art look good. The old rule that the back of the cross-stich should look as nice as the front applies throughout all kinds of art, whether it's computer programming, music, or athletic prowess.

You've got your face deep down in the last algorithm that will finish your crowning achievement of the last nine months of work. The goal is met, the task is complete, but the project is a mess.

Now its done, and you start polishing and packaging it up, writing API docs, making CD art, distributing re-usable instrument tracks for the rest of the world to see - the piece is packaged and ready to ship.

Ableton live is one of those pieces of fine art, where the purity of the design lends itself to producing beautiful musical sets, even among some of the crappiest work you've done. The layout of the clips, the coloring and grouping of the built-in tools, everything can end up being a work of art in it's own, not just the final audio dump that you'll stick on your new album.

Go download the free Múm set on Ableton's site and check out the work that they've done. If you don't have Live just download the free demo.

First off, everything sounds amazing. Any clip sounds good with any clip. Even a monkey could play a nice sounding song here. This is the meaning of the back of the cross-stitch work. The loops are clean, refined, polished, done.

They also spelled their name with inactive tracks down at the bottom of the set, and spelled their name again in midi notes in each of those tracks. I know this is poking at nerdy details, but this set just emanates the very cleanliness and artistic presentation that such a flexible tool makes possible. It's just a feeling I get, man.

This software makes me want to be creative, whether I like it or not. Not only that, but it plainly shows the way to easy and readily available creativity, without having to wade through the much of documentation and technical cribby-crap.

Ableton have nailed the solution to the music problem. They have nailed it, and then shown us what we didn't know we wanted to do in the first place. They've redefined the industry. While people preach this crap all the time, you almost never really see it.

Because of this, Ableton Live is the best piece of software out there.

Sell Out

Music people

Go download the free Ableton Live demo and then have a look at Adam Freeland's Live set on there site, here. Fabulous.

Programming people

What's the difference between music and programming anymore anyway? Bloody hell, all of today's music has been recorded, mixed, and sometimes even conceived on software.

When I went broke a month ago I sort of sold out and took back my old programming job. At least they called me before I called them!

Jackson Hole is a tough place to find a bartending job, especially in the off-season. In fact you'd be better off trying to break into the hooker scene in Amsterdam. Keeping journalistic ties with fasterskier.com has given me some opportunities to supplement my income while staying very much a part of the cross country skiing world.

This time around though, I have don't have the pressure of deadlines. I'm also working on my python-based audio scripting engine, which is a piece that only I have knowledge of and so I'm sort of working in a social hole. Oh well, at least it's money.

The good news is that I finally get to focus on Play's python-based scripting engine. We've paid a couple of guys to write some subtle portamento and legato scripts, but as of yet no one with notable python experience has gotten a chance to sit down and flush out the development workflow. It badly needs scripts written with good style and reusability concepts so that the infinite multitudes of users that write scripts for it will have a solid place to start.

Build settings and complex symbol linkage, processor optimizations, and deployment schemes still scare the nuts off me. So, the goal these days is to do most of my work in Play itself instead of actually developing the application.

The cool thing about this for Python is that we've found that the language works extremely well in a real-time audio environment. It is very fast, and imposes no noticeable CPU overhead on our professional sampling engine. I am excited to have some content and discussion-prone scripting topics on this blog, and end this needless bickering about my poor job experiences.

Oh yeah, and post some pictures of sick ski lines.

Yay

Control

OK, I'll continue on with my tirade on programming lifestyle, and computer personalities.

I'm trying to figure out how to handle subtle conflicts that arise over text chat interfaces, like AIM. The two problems I run into the most are misinterpreting other's remarks (generally for the negative, causing the other guy to seem grumpy), and the other guy answering your question or segueing so fast that you don't get a chance to finish your response.

I have no idea how to get around misinterpreting someone else's typings. This usually happens when you never get any face time with the person, and is much harder to avoid when deadlines and strict business-like requirements add pressure to the situation. Adding a spoken language barrier to the mix also adds a significant challenge.

This problem is usually always personal (meaning it's YOUR problem), and really puts your ability to give other's the benefit of the doubt to the test. After a few years of problems with this I decided I didn't have the patience for that job anymore and decided to quit.

As for the second thing, I am consistently "typed-over" by some hyper people, which is just like when someone talks over you and never lets you finish what you are saying. My current theory relates to how coders, and especially coders that "reaaally" get into their work (been there), and get so used to operating as kings in this beautifully controlled computer world that they forget that they can't control the other people they are trying to communicate with.

Hmmm.

Is a Geek a Geek?

Are geeky people into complex things because they are afraid of real productivity? After quitting my programming job I've starting taking a clean look at why geeks are drawn to complexity. Is it good or bad? Is it denial?

The other day while I was playing with my favorite music production program, I noticed something about the complexity of the interface that appealed to my geeky side.

"Oh it's so fun, look at all the little lights and level meters and mixer sliders. So cool!"

But wait, why the heck is that cool? What does all that chaos do to help me? Opening all the views and watching all the controls wizz around doesn't make my job any easier. In fact, it doesn't do *anything* for me except give me the false impression that I am associated with something successful and complicated, which maybe means that I'm super smart or something.

Not true.

I've since realized that I quit playing the violin and got into electronic music because I thought that I could get away with being creative by learning tons of geeky toys. In effect, I was trying to *engineer* creativity. So I learned the tools and specs, and even wrote and built my own tools and specs, and still my music wasn't getting as good as all the time I put into it.

What I was missing was the *expression* - that flawed human glue that gave my music life, and can only come from playing a real instrument. Because of my unwillingness to break down and practice, my music had stayed exactly as inane and lifeless as the complex computerized tools I had learned to make it.

This also applies to code. You know the apps that feel like the engineer wrote directly into the kind of experience he/she had envisioned? He/she wasn't caught up in the engine and gui communication abstraction or the details of the painting framework, but instead was able to grasp the big picture. This is just like writing a timeless song.

Now dudes and dudettes, I grew up tearing apart computers and building super complex model airplanes and lego machines just like the rest of you. But, I'm digging here because I want to get better, and I think learning about our greatest deficiencies is a step in the right direction.

Sometimes you'll be working on a component and constantly alt-tabbing between iChat, Terminal, an Xcode window, and steepncheep.com looking for skis and goggles. Sometimes you'll end up in a meeting where someone continuously talks off topic about his cool desktop widget or argument syntax modifier (don't look so guilty python guys ;)) when what you really need to do to make this app rip is optimize that boring-ass table schema.

You check your email every few seconds and love the fact that you can reply without using the mouse. When you were twelve you had five monitors and felt like the center of the matrix, controlling the world through an alternate reality. "If only they knew."

How much of this complexity is just geeky madness? No smartypants, it's not ADD, it's an unwillingness to commit to the task and possibly too much caffeine!

When do the infinite xterms stop!?! How much do you perfect your editor and build environment for hours instead of biting down and just fixing the bloody algorithm?

I've obviously been there too.

I stepped back and looked at the insane complexity of my music sequencer app and thought to myself. "What am I really trying to do here?" Is it watch smoothly animated audio levels or communicate a musical concept?

I chose the latter, pulled out my MIDI keyboard, and turned down the brightness of my display.

Since I bit the bullet and learned a few real instruments, my techno has gotten much, much better.

Red team go!

Music, Stealing, and Being Cool

Look guys, for all of you archaic rich white guys in the recording business that are freaked out about the internet ruining your profits, it's time for a face lift.

I've never bought so much music in my life after I started listening to so much internet radio. Maybe I'm some kind of freakish standout, but whenever I have a decent job (I fluctuate) and hear something I like, I almost always buy it on the spot from Amazon or iTunes. When I don't have money, I don't buy it and wouldn't buy it anyway.

Whenever I buy music I buy a downloadable copy, and I always buy it without DRM. I have to say that I really miss having a tangible collection of albums with artwork though. I really like being at home with the computer off, walking over to my music collection, picking it up, and listening to the whole album (Steely Dan's Aja, for example). Playing scratched CD's or records totally sucks, but fondling the albums is fun.

What can we learn from this, RIAA and friends? I've never spent so much money on albums, and have NEVER bought singles before being able to hear music on pandora and other internet stations. Get it out there. Keep it free, make it available, and for crying out loud CHANGE YOUR BUSINESS MODEL!

Here's some things I recently bought:

(Notice how I'm posting the links directly to the store, and notice how some of them I've probably had since the mid nineties...)

The Path

I'm having a lot of trouble getting back into programming - there's too much that bugs me about it. The duplicated compiler flags, the esoteric build configs, the unending design complexity that breaks all the rules and requires absolute perfection to feel happy about.Granted, I've always had exceedingly high standards, but why is it so hard to find the simplest joy in writing code any more?

I can't even sit at the computer for more than an hour without feeling lethargic and anxious. How will I ever work with computers again?

The only thing that sounds attractive at this point is sitting down with a plain text editor that includes the most basic editing functions, and write algorithms that include no less than 2 levels of functional displacement. Either I'm an extreme perfectionist, or finding the correct flow puts the level of productivity too low for reasonable work.

I remember the metric, like "programmers should only write 10 lines of code a day," or something silly like that. I loved whizzing through the keyboard like oil on water, as a constant flow of ideas from my mind to the machine. My fingers were faster than my brain, which felt really good.

What does this mean? Am I moving away from code and into English or something else? Am I an artist living a scientists life? I feel like this path will lead me to a higher level of skill, but am I the only one?