The Relationship Between Game Music and Live Performance

This article will explore some interesting relationships between game music and live musical performance. First, we'll get a perspective on what the classical symphony performance has in common with the act of playing a video game. Then, we'll learn about a method of turning a video game into a musical instrument for performance art.  And finally, we'll hear about a sonic toy that lets us trigger game sounds and music as a spontaneous aural performance to accompany roleplay gaming. I hope these ideas will get us thinking about the relationship between game music and live performance. At the very least, some of these ideas may tickle our creative fancy, so let's get started!

Andrew Norman's Play (Boston Modern Orchestra Project)

First, let's consider the viewpoint of acclaimed symphonic composer Andrew Norman (pictured left), who is currently nominated for a Grammy in the category of "Best Contemporary Classical Composition" for his symphonic work entitled Play. The nominated recording was performed by the Boston Modern Orchestra Project, as conducted by Gil Rose. As a composer, Andrew Norman is no stranger to accolades, having previously achieved the finalists list for the Pulitzer Prize in music in 2012 for his string trio The Companion Guide to Rome.  What's most fascinating about his symphony Play, aside from its bold and experimental approach to musical composition, is the philosophy with which it was created. As it turns out, video games played a key role.

"The piece is called Play," says Norman in an interview with the Meet the Composer program on New York public radio station WQXR.  "I was like... okay I'm going to write an expansive symphonic piece. It's in three big parts. About a week ago, maybe ten days ago, I had the idea to call them "Level One," "Level Two" and "Level Three."

If it sounds to us like a video game reference, that's entirely intentional.  "I'm not a gamer myself but I really love watching," says Norman, "and I think there's something very interesting about the way narrative is processed in a video game and the way one works through time in a video game.  The idea of trying things again and again.  This is very natural to how video games work. You try, you fail."

Norman then makes this comparison between video games and the works of Beethoven: "There's something about symphonic form that is very similar to this idea of the do-over button, and trying things multiple ways.  So in a Beethoven symphony, they present some ideas, they mix them all up, right? And then they go back and they try them all again and they get them right.  So what was gotten wrong at the beginning that created this sort of dissonance we would say, they go back and they put everything in the right arrangement.  Which to me is a very videogame-like idea."

How refreshing and inspiring to hear the worlds of classical symphonic music and video games brought together!  Andrew Norman's entire WQXR interview (which includes substantial excerpts from the Play symphonic work) can be heard here.  Also, I've included (below) a video interview by New Music USA with Andrew Norman discussing the liberation of orchestral performers to express individuality. The interview also includes two video excerpts of Play performed by the Boston Modern Orchestra Project, and further discussion by Norman of his video game inspirations.

Turning games into musical instruments: foci + loci

In an interview article published on the scholarly site Sounding Out!, the performance duo known as foci + loci explain their unusual form of musical performance art.  It's interesting to us as game composers, not only because of the music itself, but because of the musical instruments used to create it. Tamara Yadao and Chris Burke (a.k.a. foci + loci) use the creation tools of the popular LittleBigPlanet video game in order to create levels that have musical utility for live performance.

The two performance artists (pictured above) have been building their musical LittleBigPlanet instruments using the game's awesome creation tools for close to 6 years. The musical levels designed by foci + loci include both sound emitters that can be controlled by the user, and more random sounds that trigger based on many factors (including the surrounding geometry and physics).  The two performance artists chose LittleBigPlanet (LBP) as their top music application because of its flexibility and uniquely visual qualities.  "My PS3 replaced my laptop," says Tamara Yadao. "LBP replaced Ableton Live and VDMX. I think I felt LBP had more potential as a sonic medium because an interface could be created from scratch."  Here are some of those famous creation tools that LittleBigPlanet offers to its players (pictured below):

Having composed music for six LittleBigPlanet games, I'm particularly inspired by the work of foci + loci.  Using the LittleBigPlanet level building tools to create a musical instrument is an awesome way to redefine the LittleBigPlanet gameplay experience, emphasizing the creative potential of the game's software.  Plus, anything that makes the world of LittleBigPlanet more musical will always get my enthusiasm going!  I enjoyed composing music for LittleBigPlanet, but I never thought of composing music with LittleBigPlanet!  What a fun idea!

The two members of foci + loci sometimes collaborate on joint works and sometimes create their own compositions using the LittleBigPlanet game engine.  Describing the nature of these creations using the LBP creation tools, Yadao observes that "while it is often musical, sound and image are inextricably linked." Yadao adds, "The environment is the musical instrument."

This LittleBigPlanet-derived musical instrument is made possible for the most part by the "Play Create Share" philosophy that enables creativity in the game's player-base.  "Publishers (usually encouraged by the game developers) have occasionally supported the “pro-sumer” by opening up their game environments to modding and other creative uses," says Chris Burke (the other half of foci + loci). "Video game space, being a simulation, is coded mathematically," Burke explains. "We like the way the mechanized navigation of this continuous space can create a real time composition. It's what we call a "knowable" space."

What is a "knowable" space?  According to Yadao's web site, it is "the finite, or “knowable,” potential of the programmed game environment."  To apply this idea to an actual composition, Yadao created the musical work "Knowable Space" using the LittleBigPlanet game creation tools.  In this composition, the musician, in this case Yadao, moves the game's camera.  When the musician points the game's camera at sound emitting objects, they are triggered to emit specific tones or groups of tones.  At the same time, other objects fall from above and emit tones as they collide with obstructions and tumble around. These musical elements introduce a random quality that's regulated by both programmed behaviors and the physics of the environment. So now, let's see and hear the music that this "knowable space" creates:

The work of foci + loci is intriguing, because it urges us to think about game engines in an entirely new way - as creative tools for experimental composition and performance.

Syrinscape

The discipline of game audio creation can feel like a fairly abstract process.  Our sounds and music assets are folded into a matrix in which they generally exist as a subtle support for more overt visual stimuli.  For instance, in a fantasy game, when the dragon appears over a village, a player is more likely to focus on the sudden startling appearance of the creature, rather than the simultaneously triggered combat music and the cries of the panic-stricken townsfolk.  As game audio practitioners, we know that all these aural elements are combining to reinforce the excitement and believability of the experience. We can mentally encompass all the aural components that are coming together, and all the ways in which the game's programming is dynamically triggering them. But these sonic activities are happening backstage, far behind the curtain.

Audio is such an ephemeral piece of the puzzle: you can't see it, you can't touch it - there's always a sense of distance, of inaccessibility. Syrinscape may help us to mentally bridge that gap.

Syrinscape was designed to facilitate tabletop roleplaying games.  The Game Master (GM) uses Syrinscape as a tool to enhance gameplay immersion. Together with the details of the game world and the GM's powers of description, the Syrinscape app tries to help players as they mentally envision themselves in the game world.  The app supplies an evocative atmospheric element: sound.  Let's watch a video that describes Syrinscape in detail, hosted by Syrinscape creator Benjamin Loomes (pictured right).

Syrinscape may be inspiring for us as game composers, because it has the potential to take a hitherto abstract process and makes it tactile.  In fact, Syrinscape is a little bit reminiscent of a game audio engine, if such an engine had been designed to be triggered by a human being and packaged inside a whimsical user interface.  As an example, let's imagine that dragon swooping over a peaceful village. Using Syrinscape, we can manually trigger the sounds of panic and mayhem within the village.  We can watch the sound effects and music as they play, noticing how each sound cycles through variations to avoid repetition. We can make adjustments as the game action progresses, turning audio elements on and off, or switching from the soundscape of one location to another.  Here's a look at the user-interface of the Syrinscape application (pictured below).

Syrinscape could be a good way for a game industry newcomer to become acquainted with the basic concepts behind dynamic sound, and the application might also serve as a relaxing toy for game audio veterans. The application takes something highly technical (game sound), and breaks it down into something much simpler and more accessible, enabling it to be performed live.

So now we've explored some ideas that might get our creative juices flowing and redefine our thoughts about the relationship between game music and live performance.  If you have any thoughts about this subject, or have heard of any intriguing experiments or projects linking game music and performance art, please let me know in the comments!

Winifred Phillips is an award-winning video game music composer whose most recent project is the triple-A first person shooter Homefront: The Revolution.  Her credits include five of the most famous and popular franchises in video gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims.  She is the author of the award-winning bestseller A COMPOSER'S GUIDE TO GAME MUSIC, published by the Massachusetts Institute of Technology Press. As a VR game music expert, she writes frequently on the future of music in virtual reality video games.

Follow her on Twitter @winphillips.