This repository includes code that allows a Leapmotion to control a sound granulator. A python script is used to get the data from the leapmotion. The sensor’s data are then sent as OSC messages to SuperCollider where they are mapped to the parameters of a granulator.
- Linux or Mac OSX (not tested on Windows, but it will probably work with minor tweaks).
- A Leapmotion.
- The Leapmotion Desktop SDK.
- Python > 2.7, < 3.0.
- Python bindings for Leapmotion SDK. (They should come with the SDK.)
- liblo (OSC protocol library).
- pyliblo (liblo wrapper for python).
- (Can be installed using pip, which is probably the easiest way.)
- SuperCollider 3.7.
- SC3-plugins.
- SuperCollider Quarks (extensions for SuperCollider).
- dewdrop_lib
- wslib
- mmExtensions
- This is not a Quark yet, but it will become one shortly.
- Open the file
sc/main.scd
in the SuperCollider IDE (or in your favourite text editor configured to use SC). - Start the SC Server (Cmd+B or Ctrl+B, depending on your platform).
- Execute the first line of code. To do this, position the cursor at the end of the line and hit Cmd+Enter (or Ctrl+Enter). This should open a Terminal with a Python process running. If there are errors, this is most likely because some dependencies are missing.
- Leaving the Terminal window opened, execute the following section. To do this, position the cursor before the first opening parenthesis and hit Cmd+Enter (or Ctrl+Enter). Some GUI windows shoud open.
The audio of the first input of SC is recorded continuously, keeping only the last 30 seconds. Every time an onset is detected, the Onset Indicator flashes red. The three sliders can be used to adjust parameters of the onset detector.
- Gate Threshold
- This is basically a noise gate positioned before the onset detector. Any audio quieter than that threshold is ignored.
- Onset Threshold
- This is the threshold of the onset detector. It is a sensitivity control.
- Latency Compensation
- Because the onset detector takes some time to react, it may be useful to adjust the position of the detected onsets. When the prerecorded material will be copied to another buffer, the onsets will be moved earlyer by this amount of samples.
These windows have three buttons and a waveform display. The
waveform displays a buffer that can be used by the granulator.
the light grey markers are positioned where onsets were detected
(including the compensation).
- PreRecord x seconds
- Hit this button to keep the last x seconds in a buffer.
- Left Hand
- Hit this button to granulate this buffer with your left hand.
- Right Hand
- Hit this button to granulate this buffer with your right hand.
The code can be modified to get a totally different behavior, but the parameters below may be of particular interest.
- Number of channels
- By default, the granulator outputs to 2
channels. This can be changed to any number by changing the
value assigned to the
~numChan
variable in the filesc/prep/03-grasNullateur.scd
(near the top of the file). It is assumed that the speakers are equidistant from and uniformly distributed around the listener. (However, this should not prevent you from experimenting with various speaker layouts.) - Duration of buffers
- By default, two 5 second buffers and two
7 second buffers. You can create as many buffers as you want
and they can have any duration (within the limits of your
hardware and your SC configuration). This can be tweaked by
modifying the array
~recBufs
in the filesc/prep/04-liveRecording.scd
(near the end of the file). If this array is set to[1, 5, 10]
, you will get three buffers of 1, 5 and 10 seconds. The duration cannot excede the duration of the prerecorder. By default, this is 30 seconds. See next point to change it. - Maximum duration of buffers
- When the PreRecord x seconds
button, the last x seconds recorded are copied to the
corresponding buffer. The behavior of the system is not
defined if you try to copy a duration that excedes the
duration of the Pre-Recorder. You can modify the line
~lc = ~preRecorder.new(30);
in the file
sc/prep/04-liveRecording.scd
(near the end of the file) to make the Pre-Recorder buffer longer. In other words, the number in parentheses must be greater than the greatest number in the =~recBufs= array.
At this point, the python script sends only a few selected parameters from the leapmotion. It sends OSC messages to address 224.0.0.1 on port 49999. This is the All Hosts multicast address, meaning that all machines on the same network segment should receive the messages. (Some routers may not allow multicasting, though.)
The address and port can easily be changed in the python script.
- /leapmotion/numhands
- The number of hands detected by the sensor. This is sent only when the number changes. This is good to start and stop processes.
- /leapmotion/hand/left
- Number of left hands. Sent only when it changes. This is good to start and stop processes.
- /leapmotion/hand/left/orientation
- 3 floats: pitch, roll, yaw (in degrees).
- /leapmotion/hand/left/position
- 3 floats: x, y, z.
- /leapmotion/hand/left/sphere/radius
- One float: the radius of a sphere fit to the curvature of this hand. Small when fist is closed, large when hand is open.
- /leapmotion/hand/right
- Number of right hands. Sent only when it changes. This is good to start and stop processes.
- /leapmotion/hand/right/orientation
- 3 floats: pitch, roll, yaw (in degrees).
- /leapmotion/hand/right/position
- 3 floats: x, y, z.
- /leapmotion/hand/right/sphere/radius
- One float: the radius of a sphere fit to the curvature of this hand. Small when fist is closed, large when hand is open.
It may work with more than 2 hands, but I have not tested.