Procedural Audio in Unity

Our recent game Rocket Plume presented two difficult audio problems.  First was the engine noise: you spend much of the game with your engines on, and no matter what loop we tried, it became apparent that the sound was looping.  This made the sound predictable and distracting.

The second (and thornier) problem was related to the mechanics of the game.  A big part of the game is blasting away rock with the stream of pixels coming out of the tail end of your rocket.  This can happen at a very high rate, e.g. multiple pixels per frame.  Even a short digitized sound, played for each pixel destroyed, became both overwhelming and tedious.  We tried selecting from several variations of the sound, as well as randomizing pitch and volume, but it still wasn't as good as we wanted.

The solution to problems like this is procedural audio.  This is generating the sound waveform from code, as it's needed.  Procedural audio lets you create sound that never repeats, and responds immediately to what's going on in your game.

Fortunately, Unity provides some little-known but simple hooks that let you insert your own code directly into an audio processing stream.  This makes procedural audio easy and fun!

Sound in Unity comes from an Audio Source component, modified by one or more Audio Filter components on the same GameObject.  These filters are applied in the order in which they appear in the component list.  Built-in filter components include high and low pass filters, echo, distortion, reverb, and a chorus effect.

To create procedural sound, you simply create a MonoBehaviour subclass that implements the special OnAudioFilterRead method.  By doing so, you have created a custom audio filter, which you can insert into the filter chain wherever you like.  In particular, to create sound out of nothing, you just insert your custom filter right after the AudioSource component and before any other filters.  If the Audio Source has no Audio Clip assigned, it will simply stream zeros (i.e. silence) into the filter chain, which your code can change into sweet sweet sound.

But enough talk!  Let's illustrate with some examples.

Example 1 (Fake)

Let's tackle the engine noise first.  Like many natural sounds, engine noise is essentially white noise — that is, a completely random waveform — modified by some filters.  So, our first procedural audio script is really simple; it does nothing but generate white noise.

using UnityEngine;
public class EngineAudio : MonoBehaviour {
    
    [Range(-1f, 1f)]
    public float offset;
    
    System.Random rand = new System.Random();
    
    void OnAudioFilterRead(float[] data, int channels) {
        for (int i = 0; i < data.Length; i++) {
            data[i] = (float)(rand.NextDouble() * 2.0 - 1.0 + offset);
        }
    }
}

The OnAudioFilterRead magic method receives an array of float data, and how many channels are in use.  The data array represents the waveform, in interleaved format: the first sample for each channel, followed by the next sample for each channel, and so on.  This method will be called frequently (like every 20 milliseconds), with the data buffer sized as needed.  Each sample should be in the range -1 to 1; any values outside that range are simply clipped.

Because we're just generating white noise on all channels, we can ignore those details and simply throw a random number into every sample in the buffer.  The offset lets us modify the white noise a bit, essentially doing a very cheap filter right here, by causing some fraction of the samples to be clipped.

If you attach this script to a GameObject that has an AudioSource, and play the game, you'll hear a harsh static.  Tweaking the offset lets you change the sound and soften it a bit, but to really make it into something your players want to hear, you'll need to add a couple more audio filters.  We used an Audio Low Pass filter with a cutoff frequency around 800 and a Q parameter of 1, followed by an Audio Distortion Filter with the distortion level set to 0.5.  This gave us a smooth, rich, engine-y sound that was just right for our game.  And because it is generated on the fly, it never repeats or becomes predictable.

Example 1 (Real)

Confession time: the above code isn't actually what's in Rocket Plume.  We found that simply turning this engine noise on/off (for example, by activating or deactivating the GameObject) was abrupt and jarring, and the silence in between engine bursts was deafening (especially if the player has turned background music off).

So we decided that the engine noise should not cut out completely, but instead "idle" at a low background level when not in use.  Some experimenting with filter values showed that reducing the low-pass cutoff frequency would do the trick.  We extended our engine audio script to get a reference to that low-pass filter, and make this change based on a public "engine on" switch.  (There was some debate about whether this belonged in the same script as the white noise generator, but we decided to keep all the engine noise code in one place.)

The final script looks like this.

using UnityEngine;

public class EngineAudio : MonoBehaviour {
    
    [Range(-1f, 1f)]
    public float offset;
    
    public float cutoffOn = 800;
    public float cutoffOff = 100;
    
    public bool engineOn;
    
    
    System.Random rand = new System.Random();
    AudioLowPassFilter lowPassFilter;
    
    void Awake() {
        lowPassFilter = GetComponent<AudioLowPassFilter>();
        Update();
    }
    
    void OnAudioFilterRead(float[] data, int channels) {
        for (int i = 0; i < data.Length; i++) {
            data[i] = (float)(rand.NextDouble() * 2.0 - 1.0 + offset);
        }
    }
    
    void Update() {
        lowPassFilter.cutoffFrequency = engineOn ? cutoffOn : cutoffOff;
    }
}

Example 2

The next problem was the rock-chewing sound, which reflects bits of rock being blasted away by your rocket exhaust.  This is not mere ear candy; it's important feedback to the player about what's going on, and lets them know (for example) when they have blasted all the way through a platform.

Again, white noise is the core of our sound; but this time, we wanted just a tiny little burst of white noise for each pixel blasted away.  We call these little bursts "clicks," since that's what they sound like in isolation, without any filtering.  The audio script in this case has a public click count, which gets incremented (by other code) for each pixel destroyed; and then the filter code decrements that counter as it generates clicks, ensuring that the number of clicks always equals the number of pixels blasted.

using UnityEngine;
public class RockChewAudio : MonoBehaviour {
    
    public static int clicks = 0;
    System.Random rand = new System.Random();
        
    void OnAudioFilterRead(float[] data, int channels) {
        bool inClick = false;    // whether we're generating a click (true) or silence (false)
        int samplesLeft = 0;    // how many samples of that click or silence we still have to go
        for (int i = 0; i < data.Length; i += channels) {
            if (samplesLeft < 1) {
                // If out of clicks, then just generate silence for the rest of the time.
                if (clicks < 1) {
                    inClick = false;
                    samplesLeft = data.Length / channels;
                } else if (inClick) {
                    // Generate a small random silence.
                    inClick = false;
                    samplesLeft = rand.Next(1,10);
                } else {
                    // Generate a click.
                    inClick = true;
                    samplesLeft = rand.Next(2,5);
                    clicks--;
                }
            }
            for (int j=0; j<channels; j++) {
                data[i+j] = inClick ? (float)(rand.NextDouble() * 2.0 - 1.0) : 0;
            }            
            samplesLeft--;
        }
        clicks = 0;
    } 
}

Our main for-loop here is a little more complex, because we actually are paying attention to the channels.  But the basic idea is the same: we're iterating over samples, and either generating clicks (white noise), or silence.  Each of those intervals is some number of samples (2 to 5 samples per click, with 1 to 10 samples of silence in between).  When we've generated all the samples for our current silence or click, we start the next one; and when we've done all the clicks we need to do, we just fill the rest of the buffer with silence.

Just as with the engine noise, we follow this script with a Low Pass and a Distortion audio filter.  We use a much higher low pass cutoff, though (around 3000).

The result is like music to the ears... well, more like rock being chewed away by proton fusion exhaust pixels to the ears, but you get the idea.  Words can't really do audio justice, though, so check out this video of the game, where you can hear this audio in action.

I hope this article has helped you see how easy procedural audio in Unity can be.  Free yourself from the tyrrany of boring, repetitive sound effects, and add some generated sound to your next game!