Visualized Piano
I’ve been producing visuals with a MIDI enabled piano. It’s been grand to work on the audio and video simultaneously — practicing piano, playing, coding ISF/GLSL, reading the history of synesthesia & more.
Illum
I performed live at Illum Art.Tech.Music festival at The Midway, July 2019
I was invited to perform at the 2019 ILLUM Art.Tech.Music festival in San Francisco at the Midway venue. This was a great opportunity to share my visualized piano experiments with the public with a giant LED wall. The festival featured multiple new media artists and fellow A/V performers who I admire, so it was an honor to be asked to perform alongside them.
I decided to name the performance after Kandinsky — At The Boundary.
Where would the boundary between point and plane be? This approach to the external boundary — indeed, the crossing of it somewhat, the attainment of that moment when the point, as such, begins to disappear and the plane in its stead embarks upon its embryonic existence—this instant of transition is a means to the end.
I really tried to push and pull at the boundary of song, note, and sound, in the aural; color, light, and picture in the visual. I’m also keen that this work continues in a long line of artists working in the synesthetic tradition, and the physicality of the piano itself should be as core to the experience as visuals and sound. It took some doing to transport my trusty PianoDisc to the Midway, but we made it happen. I used VDMX and almost entirely ISF for the visuals in the performance, but had to teach myself some basics of live PA for the audio side. I wanted to be able to sample, playback, and play samples from the amazing Auras library in KONTAKT. I controlled my samples from a Roli Seaboard Block, and learned the basics of MPE (MIDI Polyphonic) to perform. The expressiveness of MPE really lended texture and a sense of live performance and production to the work.
Some setups were more playing driven, and some were more visuals driven with sonic overlays. Some visuals reacted very plainly and simply (Pitch to scale property, tempo to color, etc) while other were more sophisticated systems that I then reacted to when I played. I definitely found the very edge of how much I could control and just cognitively process simultaneously, although I think with some practice I could improve.
I played some covers and some original works, a culmination of the last year or so of A/V experiments. The work is still changing and evolving, there is a lot more to explore and discover. I’m still learning on the visuals side, introducing ray marching techniques and 3D into the visual side, and there is a universe to discover on the live performance side, using Ableton Live and the Roli gear.
At the Boundary
I wrote this piece after reading Kandinksy.
Only by feeling are we able to determine when a point is approaching its extreme limit to evaluate when it has entered the plane. This approach to the external boundary– indeed, the crossing of it somewhat, the attainment of that moment when the point, as such, begins to disappear and the plane in its stead embarks upon its embryonic existence–this instant of transition is a means to the end.
I also broke down and bought a decent audio interface and a few decent cardioid mics. Also thanks to my partner for filming me here with the GoPro on a school night in our living room. I’ll be showing this work and be giving a talk on all this at Binary Salon at the Midway on April 16.
Comptine d‘un autre ete — l‘apres-midi
Here I’m playing some Yann Tiersen accompanied by some shader code I wrote. Trying different documentation schemes — projections with two cameras here, a DSLR to capture the projections on the wall, and my iPhone to capture my fingers on the keys.
ShapeMaker
I’ve experimented with audio analysis and MIDI control with VDMX. This is my interface all screen captured live. This is a demo for a training series up for Vidvox. It’s an 18-part series that shows how I build visuals using ISF/VDMX for newbies. Check it out over at the Vidvox discourse for support, outlines, downloads and more.
Lightform
I’ve also had a friend bring over a Lightform and we projection mapped the piano and controlled the graphics via OSC protocol. I’m playing Metamorphosis I by Phillip Glass. It’s based on the seminal literary freak-out by Kafka.
Red Rubber Gloves
I’ve tried lots of experiments as far as capture and output is concerned. Here I’m live keying red rubber gloves with chroma key and using the image to drive feedback. This is a poor man’s depth camera. I tried to a leap motion controller to try to pull out some 3D point data from my hands but it seems to be optimized only for hands in space with no obstructions, so pointing it at the hand with keys underneath didn’t work. Might look for some other depth sensors, or buy a new phone with something built in.
Birth
Thumbing though Musurgia Univeralis yields endless inspiration and fun ideas. To quote Athanasius Kircher:
Just as the light from a fire spreads different color bodies in different directions around itself, so is sound spread through the air by moving bodies that carry its qualities. When a musical instrument sounds, if someone were to perceive the finest movements of the air, they would see nothing but a painting with an extraordinary variety of colors. This premise, for sound to be similar to light, sound must be like light in all ways, it must spread out because it has the same source.
The source here doesn’t refer to the music or light producing element – the source is that of the Devine, but this turn of phrase works intentionally both ways. Here’s a sketch of what that birth could look like.
…more to come.