The setting of bonfires along the levee is one of the more charming traditions in South Louisiana. If you’re here at Christmas time, it’s certainly worth going down there to see. That is unless, of course, you’re going out for Chinese food and a movie.
For the past few days, I’ve toyed with the Kinect sensor. In order to use the sensor, I downloaded the freenect library, as well as a freenect wrapper for both Processing (courtesy of Daniel Shiffman) and Max (courtesy of Jean-Marc Pelletier). After a bit of experimentation, I started working with the jit.freenect.grab object for Jitter.
By combining the jit.freenect.grab object with some computer vision processors (OpenCV – also thanks to JMP), I built a quick & dirty kinect-controlled synthesizer in Max. X/Y position of your hands (or head or legs) determines pitch/amplitude, and Z position triggers sound events on and off.
I’m thinking of this interface as a kind of virtual curtain. Put your hand through the curtain, and you make sounds. Move your hands around, and the sounds change accordingly. Pull them behind the curtain and the sound stop. It’s a pretty simple mapping, and I will incorporate more CV tracking to influence sound generation (direction, # of features, optical flow), and to build more complex sound objects. Right now, it’s just 16 voices of sine waves, but I’m thinking of using granular textures as well as virtual objects that will make sounds when “hit.” (Working title of this project is “Pay No Attention” as in “pay no attention to the man behind the curtain” from The Wizard of Oz).
I’ll post a video of what this looks like after the holiday break. I need to spend some time thinking about how to give this interface a non-trivial mapping, and the kind of “virtual instrument” I want to compose for. And after talking with Brygg Ullmer, I’m now thinking that this may be something that could be adapted for our laptop orchestra, the LOLs.
The good news is that I’ve had the time to become totally engrossed in working on this. The bad news is that I lose all sense of time when that happens. What is certain is that I have way, way too much fun at my job.
For Chanukah, I had bought an X-box with Kinect for the family, and I knew that people have been hacking the Kinect sensor to use in other applications and computers. The Kinect sensors sold out very quickly here, and they were gone online almost as quickly. Bottom line, I couldn’t get the sensor by itself. And unfortunately, the Kinect that comes with the X-box uses a slightly different connector (not USB), so the only real option is to get the standalone sensor, which is USB.
Over the weekend, I was in Wal-Mart (no snickering), looking for a second wireless controller for our new X-box so that Charlie and I could play FIFA11 together. As I’m browsing the electronics section, I notice that they have a couple of Kinect sensors under lock and key. I ask the clerk if they were for sale, and sure enough, they had just arrived.
So now, I’m having way too much fun playing with the Kinect sensor. I have modules built-in both Processing and Max, and will probably spend the rest of the year developing more software for it. Learning how to program this will force me to dig deeper into the video processing tools of both environments (which I’ve wanted to do for a while), as well as give me a specific direction for my work.
So far, I have learned that with a bit of planning, I should be able to make Kinect read my gestures, enabling me to use it as a performance controller. The hard part (and this is always the hard part) will be creating a performance controller that does interesting things. But doing the hard part is also what I love to do most.
It was a very weird feeling this morning, walking into my new office. I have a nice space with room for my desk, some files, bookcases and a meeting area. I also have a separate media station where I work on music/video/audio/media with a MacPro, Genelec Speakers, a Kyma Pacarana, Motif 8 keyboard, and all kinds of cool widgets and do-dads.
But this was the first time in two and a half years when I didn’t have to worry about some issue at center that required my immediate attention, when I didn’t have to figure out how to squeeze 15 minutes in for lunch, or how I was going to break away for 30 minutes later in the afternoon to pick my son up from soccer practice, and then get back to work.
In truth, my first thought upon entering my office this morning was “So, what do I do now?” I looked around, found my desk, sat down and took out my laptop. After that, I was pretty much at a loss. Did I have to call anyone? (No.) Where are those forms I have to sign so someone could take a day off? (Sorry.) Seriously, isn’t there something that absolutely has to be done right now so that the center doesn’t come to a complete stop? (Not really.)
A wave of adrenaline rushed over me (or maybe it was just the heat coming on). I was on my own agenda, which meant, I needed to make an agenda. So I did. Update on network performance project with UIUC? (Check) Update and letter of support for network arts grant proposal with UNM? (Check) Turn in keys? Meet with Stacey? Work on EM/DM blog site? (Check, check, and check). Time for lunch.
It is amazing what can get done when you’re not interrupted every 5 minutes by a phone call, walk-in, or iChat session. It’s really amazing when you don’t have to answer 30 e-mails even before you start the day. It was a remarkably productive and stressless day.
The announcement letters have been sent out and I’m very pleased to say that this year, our program will have a record representation at the 2011 SEAMUS National Conference. Five pieces have been programmed, and two papers accepted, with a total of six people going to Miami for the meeting.
Jeff, Corey and Nick will perform one of their improvisations as The Three Computeers, Yemin Oh’s audio-driven video piece Deterministic Chaos and my fixed media piece Unhinged will be presented in concert. Jesse and Corey also have works in the listening room. And Jesse and I both will be giving papers on our current research.
Why so many this year? I think it’s a confluence of events. First, our Experimental Music & Digital Media (EM/DM) program has grown rapidly in the first two years of its existence. We have four PhD students and three doctoral minors enrolled. In past years, I’ve usually only had two or three graduate students in the studio who were focused more on composing in general than composing specifically for electroacoustic media. But the arrival of Jesse Allison to our faculty has also provided a spark of energy to our program and our students are truly focused on music, technology and creativity.
Congrats to all our students for their success this year. It’s well deserved.
In music, we often talk about transitions -how things change from one state to another, from one environment to another, from one mood to another. Monday begins a transition of sorts for me, as I step down as Interim Director of the Center for Computation & Technology at LSU. After 2 1/2 years of what was supposed to be a one year appointment, I find myself glad to return to my teaching and research, while wistful that I was not able to do more while director.
It has certainly been an amazing experience. I have learned a lot about myself, about administration, and about leadership. An enterprise such as the CCT has many challenges (especially so over these past two years), and it requires a team of dedicated people to keep it going. I am most grateful to everyone who contributes to the success and achievements of the center.
Over the next couple of weeks, I plan to jot down some (hopefully) salient thoughts about academic administration, the relationship between science and arts research, tenure, and the future of computation at the university. I might even throw in some posts on my upcoming work. In any event, the purpose of these posts will be for my personal reflection and for a bit of posterity.
One of my colleagues suggested I write a book about the experience. Maybe these posts will be the start of a larger work. Who knows? All I do know is that I’ll have some time to let my brain think and be creative. And that is always a good thing.