# Continuous Gesture-Sound Interaction

## Environment

Tested using Cycling '74 Max 8.0.6
MacOS 10.14.5 and Windows 10

## Dependencies

- [Myo for Max](https://github.com/JulesFrancoise/myo-for-max)
- Mubu (install from package manager)
- [CNMAT Externals](https://github.com/CNMAT/CNMAT-Externs/releases)
- [Wekinator](http://www.wekinator.org/)

## How to
### Set-up a project
1. Open Myo Connect and connect Myo armband
2. Open the Wekinator project in the folder *continuous-gesture-sound-interaction⁩/⁨Wekinator⁩/⁨gsi_0.2⁩*. You can also create a new Wekinator fit the following settings:
   - Receiving OSC port: 6448
   - Inputs: OSC message */wek/inputs*, number of inputs 14
   - Outputs: OSC message */wek/outputs*, number of outputs 4
   - Host: *localhost*, port number 12000
   - Type: All continuous, (default settings)

3. Open the ⁨the Max project *gesture_sound_interaction_2.0* in the folder *continuous-gesture-sound-interaction⁩/gesture_sound_interaction_2.0⁩*

### Design gestures for continuous sonic interaction
1. Play the sound from the bigger play button in the [Sound Design](https://gitlab.doc.gold.ac.uk/biomusic/continuous-gesture-sound-interaction/wikis/Sound-design) pannel.
2. Select one of the first three presets
3. Choose one of those and perform the sound tracing exercise.
4. Once the you want to record your sound trace, then press the record gesture button in the [Gesture Design](https://gitlab.doc.gold.ac.uk/biomusic/continuous-gesture-sound-interaction/wikis/Gesture-design) pannel.
5. Later, find the poses related to the four anchor-points and record the by selecting the anchor point and then press the related anchorpoint related button.
6. Now you can choose one of the [machine learning approaches](https://gitlab.doc.gold.ac.uk/biomusic/continuous-gesture-sound-interaction/wikis/Machine-Learning) and explore the sound with the same or new gestures.

### Use your own synthesiser
In the case you want to use the GSI with a different synth you set your synth to receive the OSC messages set through the *OSC Communication* GUI pannel. Messages are listed in the [OSC communication page](https://gitlab.doc.gold.ac.uk/biomusic/continuous-gesture-sound-interaction/wikis/OSC-communication).

## OSC Communication
### GSI-Wekinator
#### From GSI to Wekinator
- Default host IP address: 127.0.0.1
- Default sender Port Number: 6448

Messages:

| Value                | Type               | Tag                                  |
|:--------------------:|:------------------:|:------------------------------------:|
| Myo values           | vector of 16 float | /wek/inputs                          |
| Synth parameters     | vector of 4 float  | /wekinator/control/outputs           |
| Start Recording      |                    | /wekinator/control/startRecording    |
| Stop Recording       |                    | /wekinator/control/stopRecording     |
| Train model          |                    | /wekinator/control/train             |
| Start Running        |                    | /wekinator/control/startRunning      |
| Stop Running         |                    | /wekinator/control/stopRunning       |
| Delete recorded data |                    | /wekinator/control/deleteAllExamples |

#### From Wekinator to GSI
- Default host IP address: 127.0.0.1
- Default sender Port Number: 12000

### GSI-Synth
#### From GSI to synthesiser

- Default host IP address: 127.0.0.1
- Default sender Port Number: 5567

Messages:

|Value | Type | Tag |
|:----:|:----:|:---:|
|Parameter 1| float | /tosynth/parameter1 |
|Parameter 2| float | /tosynth/parameter2 |
|Parameter 3| float | /tosynth/parameter3 |
|Parameter 4| float | /tosynth/parameter4 |

#### From Synthesiser to GSI

- Default host IP address: 127.0.0.1
- Default sender port number: 5568

Messages:

| Value             | Type   | Tag                             |
|:-----------------:|:------:|:-------------------------------:|
| Value Parameter 1 | float  | /fromsynth/parameter1/value     |
| Min Parameter 1   | float  | /fromsynth/parameter1/min       |
| Max Parameter 1   | float  | /fromsynth/parameter1/max       |
| Name Parameter 1  | String | /fromsynth/parameter1/paramName |
| Value Parameter 2 | float  | /fromsynth/parameter2/value     |
| Min Parameter 2   | float  | /fromsynth/parameter2/min       |
| Max Parameter 2   | float  | /fromsynth/parameter2/max       |
| Name Parameter 2  | String | /fromsynth/parameter2/paramName |
| Value Parameter 3 | float  | /fromsynth/parameter3/value     |
| Min Parameter 3   | float  | /fromsynth/parameter3/min       |
| Max Parameter 3   | float  | /fromsynth/parameter3/max       |
| Name Parameter 3  | String | /fromsynth/parameter3/paramName |
| Value Parameter 4 | float  | /fromsynth/parameter4/value     |
| Min Parameter 4   | float  | /fromsynth/parameter4/min       |
| Max Parameter 4   | float  | /fromsynth/parameter4/max       |
| Name Parameter 4  | String | /fromsynth/parameter4/paramName |