Sweetwater – A 3D Player Piano

Art

Music

A four-voice piano arrangement inspired by the Westworld intro, expanded to move through 3D space with time-synced visual effects for each voice. Recorded on a ROLI Seaboard RISE 25 with green screen in 4K, then processed using After Effects, Tone.js, and custom ExtendScript code to render MIDI data as animated rectangles. Mixed in Ableton with final render in Premiere.

Published

September 13, 2021

The Music and Recording

This is an arrangement I made featuring four voices. I originally got the idea from the player piano from the Westworld intro, but I wanted to expand on it — having it move through space on three axes. To make things even more compelling I wanted a time-synced effect with each voice.

After I recorded the performance on a green screen and 4K camera, I used After Effects, Visual Studio Code + Adobe plugin, and Tone.js to render the MIDI data. The mixing was done in Ableton.

Products and Libraries

I also used Premiere/Media Encoder for the final render. The ExtendScript debugger was pretty rough to use, as was walking the property arrays within After Effects to figure out how to render rectangles from MIDI data. I’ll consider alternatives. I love AE, but a good scripting interface is a must these days.

My Code

To be fair to those who wrote the extension, it’s probably not as bad as this code that drew the rectangles from the MIDI data (generated object by Tone.js). However, I haven’t found a better documented way for drawing rectangles, so I may take some time to refactor this — it’s something I’m sure other people would like to do too.

https://github.com/craigmillard/sweetwater-visuals/