Skip to content
Snippets Groups Projects
README.md 6.5 KiB
Newer Older
# Serial To OSC Exercises for Arduino, Unity, Processing
Evan Raskob's avatar
Evan Raskob committed

These are lessons on how to send and receive serial data in useful ways from Arduino to Processing, and then route that data via OSC to other programs like Unity.  These examples show how to read serial data, create strings and parse them for command and parameter patterns, and use this to build your own serial API.

To use the Unity examples you'll need to download [the OSC example project](https://github.com/thomasfredericks/UnityOSC) and of course [Unity3D](https://unity3d.com/).

For other examples, you'll need a recent version of [Processing](http://processing.org) an [Arduino](http://arduino.org).

# Communication setup

We are going to connect Arduino via serial to Pocessing, and then use Processing to connect over networks to oher software using the OSC (Open Sound Control) protocol.  Some example setups are like so:

[![example communication setups for serial to osc](images/SerialtoOSCSetup.jpg "example communication setups for serial to osc")](images/SerialtoOSCSetup.svg)

# Methods of communication

For these examples, we'll look at two methods of communication between the Arduino and another program: __passive__ and __call/response__.  In __passive__ mode, data is simply sent from the Arduino as fast as possible, with no feedback on whether or not it was received or even wanted in the first place.  It's up to the receiving software to deal with the onslaught of data, constructively.  In __call/response__ mode, the Arduino is asked (the *call*) to send something (the *response*). 

## Passive mode

Passive mode is pretty strightforward - it's a firehose of data that takes no coordination between the two programs:
![firehose](https://media.giphy.com/media/TVJzpTzptvxDi/giphy.gif "firehose out of control")

## Call/Response

Call/response takes more explaining.  In the example diagrams below, a call originates from a Processing app (via OSC) which is requesting the value of an analogue sensor atached to an Arduino.  This *call* is routed through another Processing app which acts as an OSC-to-serial bridge, tuning OSC commands into serial commands that the Arduino can process ("Serial Call diagram"):

[![serial response](images/Serial_Call.jpg "Serial Call diagram")](images/Serial_Call.svg)

Then, the Arduino *responds* appopriately via serial to the Processing serial-to-OSC bridge which is then routed via OSC back to the original app that made the *call* ("Serial Response diagram"):

[![serial response](images/Serial_Response.jpg "Serial Response diagram")](images/Serial_Response.svg)

## The API

Both of these methods depend on a common API, or a common language for describing what the calling program wants to receive and the format with which the responding program should phrase it's response.

For example, our simple API is defined as a function identifier which is a foward-slash '/' followed by a single letter, with up to 3 integers for parameters.  In the above examples, the phrase "/a,0" translates to "respond withe the value of the sensor attached to port A0 (analogue 0)".  "/a" means "respond with the value of an analogue port" and "0" means the first port, port 0 (A0).

We can build other functions on top of this simple format to flesh out our API. For example, a function command "/n" might mean "set the colour of a [NEOPixel](https://learn.adafruit.com/adafruit-neopixel-uberguide/the-magic-of-neopixels)", which could take 3 arguments for H,S,B values.  Or, a function command of "/m" might turn on a DC motor atached to the Arduino.  

Calls can work in the other direction too. Suppose the Arduino senses a button push on pin 11: it can sen the function command "/b,11,1" meaning "button pushed (1, which is true as opposed to 0, which is false) on pin 11".  Then, the Processing sketch that finally receives it can answer back with an affirmative.  If it doesn't, the Arduino could keep sending that initial call until a response is finally received.

# Files

These are the lesson files, in order of how they are introduced during the lesson:

## SerialSendReceiveStrings

A basic example that shows how to receive string (character) data from the serial port on the Arduino.

Use Arduino's Serial Monitor to send/receive data by typing it in an pressing RETURN (make sure that the dropdown at the bottom says "Newline" and not "No line ending").

Then, run the __P5SerialSendReceiveStrings__ example in Processing to try sending and receiving those same commands from Processing.  The __TEST_ON__ and __TEST_OFF__ buttons send "ledon" and "ledoff" via serial, respectively.  Look in the main .pde file for their definition.  Also, the __handleSerialCommands()__ function in the same file will process a list of all the lines of serial data (e.g. *commands*) that it has received.  It runs inside the __draw()__ function so you can directly raw to the screen inside of it.  

Right now, it's not doing much, but perhaps you can change that? 

## SerialParseTest 

This is a more advanced example that uses __sscanf__ to look for patterns in serial data an interpret them as commands.  Again, use the Serial Monitor to send/receive data to it and see how it parses what data you've typed in.  Again, run __P5SerialSendReceiveStrings__ and see what Processing is receiving.  This is the start of how we define our communications API.

## AnalogSerialAPI 

This example is a more complete version of our final communications API.  It goes along with the __P5SerialSendReceive__ and __P5OscApp__ examples.  With the two of them, it demonstrates a full call/response system where __P5OscApp__ asks (calls) for analogue values and the __AnalogSerialAPI__ on he Arduino responds back with them.  Finally, the __P5OscApp__ draws he results of those 6 analogue ports to the screen as a series of colourful rectangles.

## P5OscApp

This example plugs into the __P5SerialSendReceive__ example via OSC and demonstrates call/response.  It visualises the analogue inputs of an Arduino running the **AnalogSerialAPI** sketch.  For more on what to do with this, look at [Lesson 4 in the MA/MFA Physical Computing class](https://gitlab.doc.gold.ac.uk/physical-computing/pcomplessons2018-19/blob/master/Lesson04.md).

## VisualiseOSC

A template to use for receiving OSC messages in Pocessing and trying to visualise them inside an app, using the controlP5 libary.

## oscP5SendReceiveUnity

This example demonstrates how to use OSC to communicate between Processing an Unity.  It uses the 
Unity [OSC example project](https://github.com/thomasfredericks/UnityOSC) mentioned above and of course [Unity3D](https://unity3d.com/).