In the search for more intuitive ways for controlling sound synthesis, the hands can be the direct source of control. The Leap Motion controller provides us with detailed information about joints and gestures, and presents us with an interesting interface.


Gestures Design


In exploring the left hand, the distance between the thumb and index finger relates to the Pitch, and the distance between the thumb and the middle finger relates to the grain duration. The implementation is basic but already shows promise in becoming an engaging alternative interface. 

With the Triangle gesture, the user has simultaneous control over various parameters for synthesis. I’ve found the triangle is an engaging representation of the hand’s gesticulations and also a playful character to manipulate. It can be large or small, acute or obtuse, its plane angled in many directions.

Triangle Gesture
Selection Gesture

Selection Gesture
Using the right hand, the distance between the thumb and index finger determines the size of the subsample window. The position of the thumb also determines the position of the window within the larger sound file. I chose this gesture because the distance between the index and thumb is commonly used to represent size, and within the audio community it is common to scrub through recordings on a horizontal timeline using a window similar to this one.



direct sound control

Signal Sound Synthesis
is a free-hand gesture-based instrument for sound manipulation and performance, using the leap motion controller. This allows for direct control over sound parameters with our hands in 3d space.

Signal- a free-hand gesture-based sound instrument

Using OpenFrameworks for visualization and processing of leap data into OSC messages, we can then use these messages to control anything that receives OSC, in this case Pure Data and Max/MSP.


May 2015.