Visualized Piano

Tuesday, August 7th, 2018 | Blog, Downloads

I bought a piano off craigslist for $800 that has MIDI out. I’ve been doing semi‐regular experiments to see what I can make with it. The first thing I had to do was dust the cobwebs off my playing ability. I’m still very rusty but improving bit by bit!


At the Boundary

I wrote this piece after reading Kandinksy.

Only by feeling are we able to determine when a point is approaching its extreme limit to evaluate when it has entered the plane. This approach to the external boundary– indeed, the crossing of it somewhat, the attainment of that moment when the point, as such, begins to disappear and the plane in its stead embarks upon its embryonic existence–this instant of transition is a means to the end.

– Point and Line to Plane

I also broke down and bought a decent audio interface and a few decent cardioid mics. Also thanks to my partner for filming me here with the GoPro on a school night in our living room.  I’ll be showing this work and be giving a talk on all this at Binary Salon at the Midway on April 16.


Comptine d‘un autre ete — l‘apres-midi

Here I’m playing some Yann Tiersen accompanied by some shader code I wrote. Trying different documentation schemes — projections with two cameras here, a DSLR to capture the projections on the wall, and my iPhone to capture my fingers on the keys.



I’ve experimented with audio analysis and MIDI control with VDMX. This is my interface all screen captured live. This is a demo for a training series up for Vidvox. It’s an 18‐part series that shows how I build visuals using ISF/VDMX for newbies. Check it out over at the Vidvox discourse for support, outlines, downloads and more.



I’ve also had a friend bring over a Lightform and we projection mapped the piano and controlled the graphics via OSC protocol. I’m playing Metamorphosis I by Phillip Glass. It’s based on the seminal literary freak‐out by Kafka.


Red Rubber Gloves

I’ve tried lots of experiments as far as capture and output is concerned. Here I’m live keying red rubber gloves with chroma key and using the image to drive feedback. This is a poor man’s depth camera. I tried to a leap motion controller to try to pull out some 3D point data from my hands but it seems to be optimized only for hands in space with no obstructions, so pointing it at the hand with keys underneath didn’t work. Might look for some other depth sensors, or buy a new phone with something built in.



Thumbing though Musurgia Univeralis yields endless inspiration and fun ideas. To quote Athanasius Kircher:

Just as the light from a fire spreads different color bodies in different directions around itself, so is sound spread through the air by moving bodies that carry its qualities. When a musical instrument sounds, if someone were to perceive the finest movements of the air, they would see nothing but a painting with an extraordinary variety of colors. This premise, for sound to be similar to light, sound must be like light in all ways, it must spread out because it has the same source.

The source here doesn’t refer to the music or light producing element – the source is that of the Devine, but this turn of phrase works intentionally both ways. Here’s a sketch of what that birth could look like.


…more to come.