Description

Choose photos and images from your Photo Albums as the data source, and send touch based pixel data via Open Sound Control (OSC) to another computational device. Up to five touches on screen are detected, and the image r,g,b and intensity values at touch locations obtained. The OSC messages are sent to the IP and port number you choose; changing either value will update the target address (make sure your iDevice is on the same network as your target; with a laptop, you can create one locally, for instance). The OSC messages are of the form:

[/PhotoOSC/began, touch ID] : touch began, touch ID is a number from 0-4, stable throughout the touch action
[/PhotoOSC/moved, touch ID, x, y, r, g, b, intensity] : touch position update, x and y are in normalized co-ordinates of 0.0 to 1.0, and r, g, b, and intensity are also normalized from 0.0 to 1.0, giving the color and intensity values at the touch point
[/PhotoOSC/ended, touch ID] : touch action finished

You can now set up your own sound synthesis mappings or whatever you'd like to control.

For an example of a self-contained app based on these principles, with its own sound synthesis engine, see PhotoNoise for iPad in the Store. Have fun!