GDC 2024 Dial Up the Diegetics: Musical Sound Effects

Hello everyone!  I’m video game composer Winifred Phillips – my latest is the music for the bestselling remake of Wizardry: Proving Grounds of the Mad Overlord! (Listen and download the award-winning soundtrack for free, and vote for the Wizardry soundtrack in the public voting of the World Soundtrack Awards)

Last March I was pleased to give a lecture at the Game Developers Conference 2024 — it’s a top industry event where experts in various game development disciplines share tips and techniques related to their fields. My presentation was called “Dial Up the Diegetics: Musical Sound Effects.” Every year, after I give my lecture at GDC, I include most of the lecture content in an article series (for the benefit of those who couldn’t attend the conference). So with this article, I’m kicking off a six-part series based on my 2024 GDC presentation! In these articles you’ll find all of the discussion from my GDC lecture, supported by many of the videos and illustrations that I used to support the ideas explored in my talk.  So let’s get started!

This discussion is going to focus on sound assets, and we tend to split these into two distinct groups. On one side we’ve got composer assets – acoustic instruments, bands and ensembles, orchestras, vocals, tonal and rhythmic samples, synthetic musical sounds, etc. On the other side, we’ve got sound designer assets – environmental ambience from weather and wildlife, incidental sound effects from tools and technology, and all those sounds we make as human beings.

It’s understood that the music isn’t actually there. It’s non-diegetic – it’s detached – outside the world of the game.

But having said that, let’s now ask ourselves – what if we could tie our music more closely to the rest of the game’s soundscape? Yes, our music doesn’t actually exist in the game world, but what if our music incorporated sound design that seemed to come from that environment? Would that offer players more narrative immersion? More emotional connection? Would the music seem more intrinsically attached to the game?

Sound design for composers

Let’s now explore what happens when we game composers use the tools of sound design in our music. By integrating sound effects into our tracks, we can give our music a character and flavor that is strongly associated with the diegetics of the game. It helps our music contribute better to overall immersion. Plus, it’s a lot of fun for us as game composers!

Over the course of these six articles, I’m going to be drawing examples from the music I composed for thirteen of my previous game projects.  These include titles from some famous and popular franchises.

We’ll be considering ways in which different categories of sound design can function best from within the structure of a musical composition:

We’ll also be considering how each of these categories can make our music feel more evocative. So let’s start with the first item on our list:

Environmental diegetics

This is the ground floor of sound design – the audio of our surroundings in the game – and these audio assets can be sorted into three varieties.

The first two focus on short, discrete sounds such as the spontaneous noises that are generated by an indoor or outdoor location, and the brief sounds that would be triggered by a person or creature while moving through that unique space. The third category covers the general ambience of the area, which typically includes more lengthy audio sources.

Sound designers are naturally thinking about all three of these categories, but we game composers can use them too. So let’s start by looking at all those little noises that come from our surroundings – and here’s a good example that will show how useful they can be.

Sounds of an underwater world

Developed by Sumo Digital, Sackboy: A Big Adventure is a whimsical platformer that lets gamers jump, swing, and fly through the colorful and zany realms of Craftworld. As a member of the music composition team, I was hired to compose the music for Sackboy’s underwater adventures in the Kingdom of Crablantis. My whole job centered on musically evoking the wonder and beauty of subaquatic exploration – which is the reason why I decided to use water sounds in my music. So first I had to consider – what do we hear when we’re underwater? Something like this?

Unfortunately, that didn’t seem right – too busy and chaotic to be a good musical tool. So, what other kinds of water sounds might be useful?

Now, that’s better. I took a long recording of water dripping, loaded it into Pro Tools and edited all the individual drips into separate WAV files. Then I created a virtual instrument, applied some reverb and delay, and spread the sounds across my keyboard so that I could play them musically, like this:

That seemed to be working. But I was still thinking about those bubbles, and I started experimenting with sounds that were like bubbles, but not so chaotic – until I stumbled onto this:

It’s just the mouth of a bottle being flicked with your thumb, creating a nice ‘pop’ sound, like a bursting bubble. Seemed to work, so I gave this little ‘pop’ the same treatment that I’d given the drips, so that I could play them like an instrument:

Now that I had all these tools, I could use them in my music compositions for the Kingdom of Crablantis. Here’s what that was like:

As you can see, short, succinct environmental sounds can be really useful to us as composers.

In part two of this series, we’ll be further exploring environmental sounds, and then we’ll consider the utility of sounds derived from the animal kingdom.  Until then, thanks for reading!

Winifred Phillips is a BAFTA-nominated video game composer whose latest project is the original musical score for the video game Wizardry: Proving Grounds of the Mad Overlord(official soundtrack available now on Spotify and as a free album download on Bandcamp). Music from her latest album release, Ancient Heroes, is currently nominated for a Hollywood Independent Music Award, and is now available from the BMG record label 1 Revolution. Phillips is known for composing music for games in many of the most famous and popular franchises in gaming: Assassin’s Creed, God of War, Total War, The Sims, LittleBigPlanet, Lineage, Jurassic World, and Wizardry. Phillips’ awards include the D.I.C.E. Award, six Game Audio Network Guild Awards (including Music of the Year), and four Hollywood Music in Media Awards. She is the author of the award-winning bestseller A COMPOSER’S GUIDE TO GAME MUSIC, published by the MIT Press. An interview with her has been published as a part of the Routledge text, Women’s Music for the Screen: Diverse Narratives in Sound, which collects the viewpoints of the most esteemed female composers in film, television, and games.  Follow her on Twitter, Facebook, and Instagram.