-
balandinodidonato authoredb7e97b02
To learn more about this project, read the wiki.
README.md 4.88 KiB
Continuous Gesture-Sound Interaction
Environment
Tested using Cycling '74 Max 8.0.6 MacOS 10.14.5 and Windows 10
Dependencies
- Myo for Max
- Mubu (install from package manager)
- CNMAT Externals
- Wekinator
How to
Set-up a project
-
Open Myo Connect and connect Myo armband
-
Open the Wekinator project in the folder continuous-gesture-sound-interaction/Wekinator/gsi_0.2. You can also create a new Wekinator fit the following settings:
- Receiving OSC port: 6448
- Inputs: OSC message /wek/inputs, number of inputs 14
- Outputs: OSC message /wek/outputs, number of outputs 4
- Host: localhost, port number 12000
- Type: All continuous, (default settings)
-
Open the the Max project gesture_sound_interaction_2.0 in the folder continuous-gesture-sound-interaction/gesture_sound_interaction_2.0
Design gestures for continuous sonic interaction
- Play the sound from the bigger play button in the Sound Design pannel.
- Select one of the first three presets
- Choose one of those and perform the sound tracing exercise.
- Once the you want to record your sound trace, then press the record gesture button in the Gesture Design pannel.
- Later, find the poses related to the four anchor-points and record the by selecting the anchor point and then press the related anchorpoint related button.
- Now you can choose one of the machine learning approaches and explore the sound with the same or new gestures.
Use your own synthesiser
In the case you want to use the GSI with a different synth you set your synth to receive the OSC messages set through the OSC Communication GUI pannel. Messages are listed in the OSC communication page.
OSC Communication
GSI-Wekinator
From GSI to Wekinator
- Default host IP address: 127.0.0.1
- Default sender Port Number: 6448
Messages:
| Value | Type | Tag |
|---|---|---|
| Myo values | vector of 16 float | /wek/inputs |
| Synth parameters | vector of 4 float | /wekinator/control/outputs |
| Start Recording | /wekinator/control/startRecording | |
| Stop Recording | /wekinator/control/stopRecording | |
| Train model | /wekinator/control/train | |
| Start Running | /wekinator/control/startRunning | |
| Stop Running | /wekinator/control/stopRunning | |
| Delete recorded data | /wekinator/control/deleteAllExamples |
From Wekinator to GSI
- Default host IP address: 127.0.0.1
- Default sender Port Number: 12000
GSI-Synth
From GSI to synthesiser
- Default host IP address: 127.0.0.1
- Default sender Port Number: 5567
Messages:
| Value | Type | Tag |
|---|---|---|
| Parameter 1 | float | /tosynth/parameter1 |
| Parameter 2 | float | /tosynth/parameter2 |
| Parameter 3 | float | /tosynth/parameter3 |
| Parameter 4 | float | /tosynth/parameter4 |
From Synthesiser to GSI
- Default host IP address: 127.0.0.1
- Default sender port number: 5568
Messages:
| Value | Type | Tag |
|---|---|---|
| Value Parameter 1 | float | /fromsynth/parameter1/value |
| Min Parameter 1 | float | /fromsynth/parameter1/min |
| Max Parameter 1 | float | /fromsynth/parameter1/max |
| Name Parameter 1 | String | /fromsynth/parameter1/paramName |
| Value Parameter 2 | float | /fromsynth/parameter2/value |
| Min Parameter 2 | float | /fromsynth/parameter2/min |
| Max Parameter 2 | float | /fromsynth/parameter2/max |
| Name Parameter 2 | String | /fromsynth/parameter2/paramName |
| Value Parameter 3 | float | /fromsynth/parameter3/value |
| Min Parameter 3 | float | /fromsynth/parameter3/min |
| Max Parameter 3 | float | /fromsynth/parameter3/max |
| Name Parameter 3 | String | /fromsynth/parameter3/paramName |
| Value Parameter 4 | float | /fromsynth/parameter4/value |
| Min Parameter 4 | float | /fromsynth/parameter4/min |
| Max Parameter 4 | float | /fromsynth/parameter4/max |
| Name Parameter 4 | String | /fromsynth/parameter4/paramName |