Hybrid Dynamic-Diegetic Music for Game Music Composers (From Spyder to Sackboy: GDC 2021)

Hi! I’m video game composer Winifred Phillips. Welcome to installment five in my series of articles based on my lecture, From Spyder to Sackboy: A Big Adventure in Interactive Music. In delivering my presentation at this year’s edition of the popular Game Developers Conference, I based my lecture content on my experiences composing music for two projects in simultaneous development at Sumo Sheffield – Sackboy: A Big Adventure for PS5/PS4, and Spyder for Apple Arcade. (Above you’ll see a photo from one of the sections of my GDC 2021 lecture in which I’m discussing the Spyder project).

The music design for these two games included multiple dynamic systems that were both complex and ambitious in scope. While they both relied on some of the most tried-and-tested strategies for musical interactivity, they were also quite innovative in their own distinctive ways. While composing music for these projects, I had the opportunity to see how flexible dynamic music models can be. I learned a lot from the experience, and it was really interesting to explore the similarities and differences during my GDC 2021 lecture!

If you haven’t had a chance to read the previous installments of this series, you can learn first about how dynamic implementation can benefit from the inherent segmentation of traditional song structure when horizontally resequenced. Then you can pursue the possibilities of horizontal resquencing further with a discussion of the importance of dynamic transitions. After that, you can examine the dynamic possibilities afforded by pure vertical layering. Once you’ve completed that article, you can take a look at how horizontal and vertical structures can be combined into hybridized forms.

With all this in mind, we should remember that a music design doesn’t need to rely strictly on one approach or another. Combining these strategies yielded effective results for both Sackboy: A Big Adventureand the Spyder game, which I was working on at the same time.

In the article that immediately preceded this one, we focused on Sackboy’s execution of the hybrid horizontal-vertical structure. The implementation for Spyder, though, was quite different, so let’s take a closer look at that hybridized dynamic approach.

During one of Spyder’s more stealthy levels, Agent 8 infiltrates the enemy’s high-security War Room. While pursuing his mission to obtain some top-secret documents, Agent 8 accidentally turns on a desk radio, triggering some diegetic music to start. My 60s-style lounge jazz composition begins playing from the radio, and this keeps going throughout the rest of the mission:

Diegetic music – like my lounge jazz track from the War Room radio – is considered to exist inside the fictional narrative, whereas non-diegetic music, or dramatic underscore, is outside of it. Now that we have that continuous diegetic jazz music playing, does this mean we can no longer have dramatic underscore? Not at all! We just have to apply some creative solutions.

The Spyder audio team at Sumo and I solved the problem with a hybrid dynamic music system that included both horizontal segments and vertical layers. First, a vertical layer would start playing that continuous jazz music from the radio. Then, the dramatic underscore would come in on top, delivered in over thirty horizontal segments. Like the famous ‘success melody’ from Sackboy that we discussed previously, these music segments were structured to blend with the other music that was already playing. So – every measure of the radio music had a corresponding measure of dramatic underscore.

These underscore segments would only play when triggered, so they’d usually play in different horizontal sequences. This helped the music stay fresh as gameplay progressed. Let’s check out what that was like:

As we can see, this interactive system from theSpyder War Room level accomplishes many goals. The horizontal segments indicate progress through the level. They create musical variety, and they reward players for their expert spy skills. Plus, the vertical layers allow us to merge the diegetic and non-diegetic music together. Both the Sackboy and Spyder games used the hybrid horizontal-vertical system very effectively, and in very different ways.

There is another goal that dynamic implementation can address, and it’s a much more practical consideration than any of the others. Dynamic music can stretch one piece of music so that it’s doing the jobs normally done by several tracks. Dynamic systems can make one piece of music take the place of an ambient track, a puzzle-solving track, and a high-energy action track. It’s a great way for development teams to optimize music coverage for their games. Let’s see a couple of examples.

In a previous installment of this article series, we checked out the “Shoo-Bee-Doo Hullabaloo” music from Sackboy: A Big Adventure. While we already looked at the big boss-fight music at the end of the level, that same music starts the level in a very low-key way – it’s pretty understated and calm. And that same musical composition builds over time from the most relaxing textures to the greatest dramatic intensity. It accomplishes this by virtue of a procession of ever more dramatic segments, and a mix that progressively increases in complexity and weight.

By the time we get to the end of the level, it’s gone from a peaceful ambient texture all the way to a rousing orchestral finale. Let’s check out how that transformation occurred:

So now we’ve looked at how horizontal and vertical elements can be combined into hybrid forms. These options allow game audio teams to creative dynamic implementation schemes that react to gameplay in lots of awesome ways. In the final article of this six-part series, we’ll be taking a look at the last item on our list of implementation structures:

What happens when we try to capture the best qualities of purely linear music, and retain those advantages from within a hybrid linear-dynamic music system? We’ll explore that idea in our next article. Until then, thanks very much for reading!