November 20, 2011

Tones is an interactive software piece that generates musical patterns based on colour data from a live camera input. Visuals are fed from the camera and downscaled to a 10×24 pixel grid, where the darkest pixel of each column is marked in orange. Sound is continuously generated and reiterated based on the height of these marked pixels, where the height is either proportional to the pitch of a note or the volume of a track.

The work attempts to create a codependent relationship between two distinct forms of sensory communication, audio and visual, looking at how data can be repurposed and translated from one function to another. Marking the darkest pixel of each column enables users to plainly see how their presence affects the data provided to the computer, but they must uncover on their own how their position and movements contribute to the generated sound. Users are invited to explore how their interaction with the piece contributes to the sound generation without being given any definitive guideline or affordance, in turn creating both a generative output and generative user interaction.