Using Max 8 as a synthesizer for Unity with UnityOSC

Note: The files and folders used in this document are available at https://vault.sfu.ca/index.php/s/RWBi7AlSFLqTS2r

Note 2: A demo of this integration can be seen here:

youtube

Overview

This tutorial will give a basic example of how to use jorgegarcia's UnityOSC to send individual notes to Max 8, which Max then interprets and synthesizes into sound.

UnityOSC

As mentioned, we'll use UnityOSC to send OSC signals from Unity to Max 8. The first thing we need to do is set up UnityOSC in Unity

The first step is to install UnityOSC by following it's instructions from github: https://github.com/jorgegarcia/UnityOSC

Once UnityOSC is installed, we need to add a single line to the Init() function in the provided OSCHandler script (If you put the repo into your /Assets folder, OSCHandler will also be in the base Assets folder):

This code creates a client by name ("Max"), assigns ip address of 127.0.0.1 since we're running this locally, and assigns the port.

I picked 6000 as a port as an arbitrary value, feel free to use a different value if you'd like.

We then add the code:

OSCHandler.Instance.Init();

Which may be added anywhere, but I put it in the GameManager's Start() function.

And with that, UnityOSC is now ready to use.

Sending Message from Unity to Max 8

There are two ways that we can send messages from Unity to Max 8, and they're almost interchangeable. The first way is sending a single value, for which we would use the code:

OSCHandler.Instance.SendMessageToClient("Max", "PrependMess", Value);

where Max refers to the Client name that we provided in the CreateClient function

PrependMess is a little more difficult - when Max 8 receives the code, whatever string is given as PrependMess will prepend to the passed Value. Note that UnityOSC's api will refer to this value as "address", which is equally confusing to me.

and Value is the single value that we want to send.

Another way to send data is to send a list of values, using the same syntax:

OSCHandler.Instance.SendMessageToClient("Max", "PrependMess", List{Values});

We will be making use of Max 8's makenote object, and will therefore provide pitch, velocity, and duration.

Our code uses a few functions to send a note to Max. We won't get too into the weeds of the generative logic here, but each voice selects a pitch, velocity, and duration every few seconds.

First, the setup for each voice.

Each voice has the following public variables:

To prevent things from getting out of hand, we have all OSC data sent by the GameManager object, and each voice must have a reference to it.

instrumentName is used to fill the PrependMess field and let Max know which insturment is playing. In our case, we assigned the insturment names Mel1, Mel2, Mel3, Rhy1, Rhy2, Rhy3, Bass.

Each voice plays a pitch every few seconds. Once we have selected a pitch, we call the following functions:

In SingleNote (the script that is assigned to each voice)

In GameManager:

This function is fairly simple - the voice sends its own identifier name, as well as a pitch, velocity, and duration. The GameManager places the pitch, velocity, and duration into a list of floats to send via OSC, and then sends the message via OSCHandler.

Now let's look at the Max 8 side, to see how the data is interpreted.

Max 8

The first step in Max receiving the data is to use the udpreceive object, with the argument of the port that we set in the OSCHandler, in this case 6000. If you want to experiment, you can hook up the udpreceive 6000 object to a simple print or message object and observe the data.

As an example, let's pretend that Mel1 is playing a forte C4, midi note 60, for 2.5 seconds. Max receives a message with the following: Mel1 60 110 2.5

Obviously, we need to break that up a little to use it.

The first step is to use the route object, whose help file is here:

Route essentially lets us separate out the messages based on the PrependMess that accompanies the OSC data. We use it as follows:

As we see, we simply attach the udpreceive object to the route objects. Each of the arguments is one of our previously assigned instrumentNames from Unity.

In our example, this would send the message 60 110 2.5 out of the first outlet. Note that if you are using noteout to synthesize using MIDI, you can instead use a poly object, but in this case we use FM Synthesis.

Once the data is routed, we need to interpret it, which we do as follows:

This takes place in an abstract FMSynth patch, so that we only design the instrument once.

This takes the incoming message and splits it into three different messages: 60, 110, and 2.5. Because the third argument of makenote is provided in milliseconds, and our message is provided in seconds, we need to convert the duration.

Makenote sends a pitch and velocity, and automatically reduces the velocity to 0 when the duration has completed. We use the FM Synth from the Max tutorial (https://docs.cycling74.com/max5/tutorials/msp-tut/mspchapter11.html), with two alterations. Our full, extremely messy patch can be seen here (Note: The version that you download may be even messier, I forgot to save some alterations) :

The alterations that we make are to allow for volume control and duration control, as the original patch would simply play a pitch forever. We also added the nslider object just to give some visual feedback about our pitch.

The added gain~ object takes both signal and an integer as input - the signal is scaled to the integer value. We take the value directly from the makenote object. In our example, the gain~ slider will move to 110 when the note is sounded, and 2.5 seconds later will move to 0.

This is then multiplied by another volume slider, so that we can set the volume of each voice while the gain is programmatically controlled. This is probably not the best way to do this, but it works.

Each instrument simply outputs directly back to a global ezdac~ object. The full patch is as follows:

Each bpatcher is an instrument that we've shown.

And that is the entire patch for sending Unity messages to Max via OSC. Let me know if yout have any questions and I'll hopefully clear things up.