GitLab now enforces expiry dates on tokens that originally had no set expiration date. Those tokens were given an expiration date of one year later. Please review your personal access tokens, project access tokens, and group access tokens to ensure you are aware of upcoming expirations. Administrators of GitLab can find more information on how to identify and mitigate interruption in our documentation.
The max patch works in communication with Wekinator.
1. Open Wekinator
2. From Wekinator, open `./Wekinator/gesture-sound-interaction.wekproj`, and press start listening.
3. Open `./gesture-sound-interaction/gesture-sound-interaction.maxproj`
3. Open `./gesture-sound-interaction/gesture-sound-interaction.maxproj`
4. Connect Myo, and stream data.
5. Do calibration processed if prefered (optional).
### Authoring
The authoring mode allows you to create static-sounds gesture relationships. You can set sound paramenters via the SCP synth, and then train the neural network (ANN) model using Wekinator.
### Sound-Tracing
The sound tracing mode allows you to create time-variant sounds-gesture relationships.
In the `envelopes` sub-patch you find different evelopes, one per each parameters of the syntesiser. For each parameter you can establish the value of each of the four points of the envelope, numbered as in the range [0, 4]. Evelopes points can be edited by dragging each point within the table, or by selecting a point (`Select Point`) and editing the `Point value`. The duration of the envelope can be also edited by editing the `Duration` value. The enveloped can be played by pressing `Play envelope`.
In sound tracing mode the the sound gesture relationship can be established by using two algorithms, ANN and HMM.
The ANN implementation uses Wekinator, instead the HMM on uses the Mubu library.
When training the ANN model, it is possible to record the whole gesture or segments of it togheter with the time varying or static sound parameters, (`Rec. Whole`/ `Rec. Segments`).
When recording the whole gesture and evelope evolution, there is no need to take the `Window` parameter.
The sound tracing mode allows you to create time-variant, sound-gesture relationships.
In the `envelopes` sub-patch there are different envelopes, one per each parameter of the syntesiser. For each parameter you can establish the value of each of the four points of the envelope, numbered in the range [0, 4. Envelope points can be edited by dragging each point within the table, or by selecting a point (`Select Point`) and editing the `Point value`. The duration of the envelope can be also edited by editing the `Duration` value. The enveloped can be played by pressing `Play envelope`.
In sound tracing mode, the sound-gesture relationship can be established by using two algorithms, ANN and Hidden Markov Model (HMM).
The ANN implementation uses Wekinator and the HMM uses the Mubu library.
When training the ANN model, it is possible to record the whole gesture or segments of it togheter with the time-varying or static sound parameters, (`Rec. Whole`/ `Rec. Segments`).
When recording the whole gesture and envelope evolution, there is no need to take the `Window` parameter.
In the case of recording segments (`Rec. Segments`), then you might want to edit the length of the segments to record in ms (`Segment`). This number corresponds to the windows size of recorded samples.
In both modalyties, you have to enable the ANN (`Enable ANN`), then play the envelope (`Play Envelope`). The software will automatically record and train the model for you. Before running and playing sounds, please check that the training phase has been succesfuly completed.
In both modalities, you have to enable the ANN (`Enable ANN`), then play the envelope (`Play Envelope`). The software will automatically record and train the model for you. Before running and playing sounds, please check that the training phase has been succesfuly completed.
To train the HMM, you have to adopt the same approach; enable HMM (`Enable HMM`) and then play the envelope (`Play Envelope`). The software will train the model at the when the audio file has been played.
After training the model, you can choose if scrubbing along the whole file (`Scrub Envelope`) or to scrub along points (`Scrub Points`). To access the HMM data, please open the HMM subpatch.
...
...
@@ -43,6 +46,7 @@ Moreover, the sound tracing mode can be also used as alternative to the authorin
After recalling any point of the envelope, it is possible to recod and train Wekinator with such parameters. When running wekinator, you will be able to explore the space between the choosen SCP paramenters-gestural associations.
### Sound Design Mapping
The subpatch `SoundDesign-Mappings` contains all messaging system to interface the SCP synth with Wekinator.
| # | Wekinator ANN | Synth control | Parameter |
...
...
@@ -65,18 +69,22 @@ This mapping can be deleted by sending the same message with parameter `0`.
It is possible to set the minumum and maximum range values indicating the Wekiantor ANN, the min or max strings, respectivelly for the minimum and maximum of the range, i.e. `6448weki01 min -1.;`
`6448weki01 max 3.;`
The data stream from/to Wekinator can be limited and interpolated with a message that contains reference to the Wkinator ANN, the string `speetlim` for limiting and `time` for the interpolation and the time in milliseconds.
The data stream from/to Wekinator can be limited and interpolated with a message that contains reference to the Wkinator ANN, the string `speedlim` for limiting and `time` for the interpolation and the time in milliseconds.
### EMG calibration
Without any calibration, the output data are the extracted features, in this case the bayesan filter. the calibration process normalises data from each channel between a range [0, 1].
-**min:** press min and relax your arm until you dont see stable values
-**max:** press max and contract your muscles performing different movements until you dont see stable values
-**run:** after calibration hit run!
-**reset:** resets calibration values. *NB: if you want only to update the min or max values of a specific emg channel you can just press min or max again, and relax/contract muscles relative to that channel. In this way you won't loose calibration data for other channels.*
### IMU calibration
-**calibrate:** set the origin of the three signals around the 0 value.
-**reverse:** do the inverse of the euler angle on the x, y, or z axis.
