Instrumentation: The Game Composer's New Challenge

There are a lot of hot topics out there now, that game music is "coming into its own"--surround sound; the ever-present live orchestra; the latest pop license. However, whether you're doing surround sound or using a hit band or the Boston Pops, one thing stays the same across it all: your instrumentation. The voices of your art.

Just why is instrumentation important? Take a listen to some of the most respected game soundtracks:

Wipeout XL
Icewind Dale
Indiana Jones and the Emperor's Tomb
The Dig
Myst 3

Each of these uses instrumentation effectively and dramatically. In some cases, such as The Dig, they deviate from the norm in terms of what is currently popular, and generate unique qualities all their own. It is this sort of effective deviation that will keep games unique as a media.

When guys like me started in this business 10 or more years ago, instrumentation was easy by comparison. You rarely had more than 9 to 11 voices of polyphony and not more than a bank of 127 possible instruments with the old General MIDI tracks. If you really wanted to limit your palette for quicker and easier composition you might have used an AdLib or SoundBlaster's FM synth. For consoles you had a tiny fraction of the available memory to play with. Now, only the imagination and the tools at hand dictate instrumentation, whether they're sampled, recorded live, or synthesized from scratch. A new challenge faces game music composers, and that is choosing our weapons.

When faced with limited instrumentation more attention was paid to composition. You chose what sounded good from your soundcard or console's meager capabilities and created what pleased you.

Technological limits

Take a listen to this sample from a GameBoy Color track. Created with a 5 PSG (Programmable Sound Generator) voice system that could generate a triangle, sine, or square wave (with even 1-bit digital sampling), it even had less capability than the Commodore 64's SID chip, which came out in 1982.

Not bad for something that only needs to provide additional atmosphere for something already perceived as an electronic escape. There's no denying that it's simple, it's fairly catchy, and it's cute, analyzing it on an emotional level. What's even more important is that instrumentation of this sort was brand new, in all of music history. Nothing like it had ever come before, and "it wasn't in the public eye the way pop music was" arguments aside, Super Mario Bros. is still the best selling video game in history, having sold over 40 million copies. Not many artists have gone 40 times platinum--and not many film soundtracks have either.

Now we move into the second age of game music. (Well, let's just say it's just one possible second age.) If we went by technological advancements alone we could well have over 15 ages of game music. In any case, at this point we're moving away from simple distraction. We're moving towards pushing that distraction towards an even more impressive visual and aural treat. Dare we say we're moving towards altering the player's reality?

Another sample track, this time a MOD, is a tribute to the advanced technology of the Amiga home computer. This piece was written in 1997. There are 16 channels of PCM sample based playback, totaling 1MB of space per song. Not on par with the greatest studio records of the 1990s (dear God no…not with 8-bit samples), but it conveys far more emotionally and dramatically doesn't it? Plus, it still doesn't sound much like anything else selling in record stores or playing in movies. Nevertheless, rather than beeps, we're getting string-like sounds and more layered synthesis. Rather than single sine wave based thumps, we're getting bass drums…yes, the same bass drum sample each time, differentiated only by volume and pitch controls within the track.

Now we move to the modern age, the new renaissance of game music. All of a sudden, record producers in the USA are noticing game soundtracks and feature film composers are turning their heads to a more successful media. The orchestra simultaneously becomes attainable and a symbol of the power that game music can achieve.

Twenty-five members of the Austin Symphony lined up with Paul Baker as our orchestrator and conductor for this sample piece, and we recorded it in the Fire Station studio at Southwest Texas University. It wasn't Hollywood, but it was still a surreal experience. The icing on the cake was that it was mixed in 5.1, with five discreet speakers and a subwoofer delivering the instruments in relative positioning to the listener. 5.1 and other speaker configurations are opening up new doors for presenting music, but they're also posing new challenges. First, the composer is often not the engineer, so one must add a new task to the budget if one deems 5.1 as important to the game. Second, one is not usually inside the orchestra pit during a concert, but does that mean they shouldn't be? Would that be the best way to present something if a composer could have done so for concerts of old? For this piece I said "what the hell" and it worked out very adequately, but I still would have preferred to have had a professional engineer do the mixing. I deserve a lot of guffaws from my most respected colleagues for the mastering job.

Regardless, now that we've heard the magnificence of an orchestra, doesn't it sound like something we've heard before? It sure does...a few folks told me it reminded them of the cartoon "X-Men" theme, even though I never saw the show. Lesson? The uniqueness and cult phenomenon of our art has faded, but let's not shy away from that. Let's attack it instead.

New composition

We game composers are now facing what every composer faces--the broader the palette, the more difficult the choices. Now that we're unrestricted and free, we burst upon a field already teeming with activity rather than taking heart in the "making more with less" skill with our little band of colleagues. The question now becomes, "how can I make my game music unique, while still appealing to the masses?" To answer this question let's explore some possibilities, starting with composition, since composition and instrumentation go hand-in-hand.

The orchestra is so popular, across the board, in the entertainment industry because it represents a vastness and fullness of sound that has yet to be challenged. The choir also serves this purpose. Before we discard the instruments, let's think about how they are used. Orchestral instrumentation, using standard tonal harmonic techniques, gets used typically because people are trained in typical compositional contrapuntal styles in most institutions. Even outside the academic circle, production managers who lean towards the "if it ain't broke, don't fix it" common sense can pigeonhole instrumental techniques. An example of this is simple When you think "theme" what comes into your head? A single instrument (or group of instruments) playing a melody, right? Of course, a "theme" has been defined by pop culture as being exactly that--but it doesn't have to be. While all commercial music doesn't follow these limitations, a good portion does. This isn't always bad, but it certainly means if you were to listen to 20 feature film soundtracks in a given genre in a given year, 15 of them would sound awfully similar.

A great example of a mainstream popular work that goes against this tradition is the soundtrack to The Matrix. No, not the songs…the score. Don Davis decided against tradition and went with more modern techniques (i.e. Stravinsky, Philip Glass). The author will be the first to admit that he really doesn't like that much in the way of modern techniques, but The Matrix conveys a perfect futuristic musical representation of the film using ancient instruments. You have to admit that's a pretty impressive feat. So, what did he do? He used dissonant chords, 12 tone scales, melodic and harmonic inversions, repetition, tritones--the list goes on. It would require a far longer article to explain what these things are, but ask me if you don't know, and I'll be happy to give you a demonstration on a single instrument basis. In the meantime, pick up the soundtrack, watch the DVD if you haven't yet, or if you're feeling thrifty, go to Amazon.com and listen to the samples. He used these alternative methods interweaved with the same catchy single instrument melodic themes we're so fond of, and it worked very well.

Now let us explore a different approach: changing the instruments themselves.

Instrumentation examples

These days the orchestra is the popular kid on the block, for the reasons I've already given. However, there are also some excellent examples of non-orchestral instrumentation. Take this simple example.

The combination of piano, dulcimer, and a dab of clay drums give this a different sound than the usual five instrument rock setup or the standard techno sound. Use ethnic instruments to your advantage. There are a huge number of them out there, and they can all serve a plethora of purposes. Deus Ex: Invisible War uses Indian instrumentation (dumbek, tabla, sitar, sarod) in its Cairo maps. Inappropriate for reality, but it generates the right kind of feel for a Middle Eastern country in the far-flung future.

Should you really want orchestral as your musical toolbox there are plenty of unorthodox instrumentation techniques to use. More importantly, there are ways of NOT following orthodox methods. Listen to the following samples of a swell, which can be used for a cinematic moment during a cutscene or during gameplay, whatever the situation demands

This first swell uses a number of standard techniques. Strings, horns, tympani and snare are the instrument groups, and these are what everyone uses initially as the easiest instruments to generate a full sound. Unfortunately it sounds like a million other soundtracks, doesn't it? Wandering notes within a single chord and key structure, overall bang at the end. This was played with canned synth presets, but even if it had been played by the London Symphony, the effect would be 90% identical in terms of the message it sends: canned as well.

The second swell is different. It uses bassoon, a clarinet, and a single French horn to convey a dissonant melody in the midst of the chord swell with the string section. It also uses a tam tam instead of tympani, no snare, and no cymbal. It also flows much better, since the strings decrescendo gradually and then crescendo towards the very end. The result is something just as emotionally effective but also more specific--it presents a dissonance of emotion with that melody line, and not letting the listener sit back comfortably listening to their pre packaged chords that have been employed in far more pieces for hundreds of years.

As to synthesized instrumentation, that's a little more difficult. For those of us who toyed with simple waveforms it was easy to build a sound bank. Now, with the advent of such products as Reaktor by Native Instruments, and my personal favorite, Absynth, things get a bit more complex. Yes, you have to spend days or even weeks building a sound set, but the result of the Absynth composition is worth it. I urge all fellow composers to truly think about the message they're sending when they sit down to play with the usual factory patches. Granted, time constraints may not make this possible, but there's always time in-between projects. Devote that time to finding that elusive new sound you never knew could move you, rather than being comfy with a Resonatix copy that reminds you of "Tom Sawyer". (I love that song, I just don't use that instrument anymore unless I want it to remind me of "Tom Sawyer".) This synth-based clip is an example of what I mean. Synth patches like this are still in use today, and in some cases I'm sure they're very apt, but there are other worlds to explore.

Let's move on to examples in the public domain. "Lux Aeterna" by Ligeti, from the film 2001: A Space Odyssey freaked out everyone who heard it. If you haven't heard it yet, head to Amazon.com, look it up, and hear the sample. Almost nothing like it has been done before or since. It was revolutionary and effective. Using a choir in this sense for science fiction? How bizarre…let's scratch our heads over that one at the production team meeting.

Well, guess what. Marty O'Donnell followed suit by putting Gregorian chant into Halo. No unintelligible Latin accented bits, no soft Casio sounding "oohs" and "aahs"… he broke new ground, and he did it with a game soundtrack. Can't beat that.

Let's take another look at instrumentation in games that has worked well. The adventure category is the ripest field for experimentation:

Morrowind used orchestral elements in conjunction with synthed ones. The theme itself is almost too simple, yet incredibly dramatic by using the "Bolero" approach. The theme begins by being stated using a simple flute and synthy woodwind harmonic support. The volume is very piano (soft). It then is restated with sweeping strings in mezzo piano (medium soft), slightly louder, with the downbeat accented. The third and final restatement of the theme is all out forte (loud) with horns supporting the strings and more percussion. It doesn't modulate the way a piece like Star Wars does using a form of ABA (A theme, B theme, A theme repeated), but the use of instrumentation negates this.

Myst 3 also did this with pop vocal elements (see the Quicktime "Making the Music" trailer). The main theme itself goes from subtle female pop vocal ("find your way back to me") with a sultry and mysterious undertone to a very strong choral theme using rather standard "Dies Irae" Mozart presentation. Regardless, the switching between these two styles is innovative, blending tried and true techniques with modern day popular vocal thematic usage.

Read about Chance Thomas's latest work, with some very impressive instrumentation alternatives in the Gamasutra article about Lord of the Rings. In this case, Chance and the composers he worked with designed the instrumentation as thematically as possible, assigning specific non-standard acoustic instruments to each race of creature as well as each character.

Uru, the latest in the Myst series, even has a track from the renowned Peter Gabriel. While music licensing has mostly been about more urban and Billboard based pop trends, this represents linking alternative pop to a completely different kind of brand, and it works.

Deus Ex: Invisible War followed suit with this trend. (Check out the website www.deusex.com for a download of the entire soundtrack.) In addition, we licensed tracks from the Kidney Thieves. Perhaps not as popular as Gabriel, but uniquely fitting for the genre.

Licensed music

Let's not ignore at least a few other genres for fullness of examination. Take sports and driving games, for example. From the old Nintendo favorite 10 Yard Fight and Rad Racer to the latest and greatest NFL licenses by the sports publishing god EA, as well as the ever popular Need for Speed franchise, the new subsidiary EA Trax is finding popular and otherwise appropriately grand scale music to incorporate into its titles. The instrumentation used in these pieces is the same as that which the most popular musicians deem fit to go platinum, so yeah, they should sure be effective.

In some titles this works incredibly well. Take Wipeout XL from Psygnosis, with electronica artists such as The Prodigy gracing its tracks. These are linked to the theme of the game very appropriately and have set an example for other games of its kind to follow, should they wish to license. Considering the marketing angle, electronica geeks might have been bored by such a move, but the market at large less so.

In some titles, it doesn't work so well. Tiger Woods 2004, with songs from such artists as DMX and Rooney, has been hailed as a fantastic game for the most part, but its seemingly randomized assortment of pop favorites have been commented on as being jarring and inappropriate by reviewers and gamers alike. While the music itself would have appealed to the age group and market segment of the game, the random assortment of it left them baffled. It was a bold move…most would expect a baroque chamber piece to be played at a golf tournament while any onlookers younger than 35 yawned. Nevertheless, licensing for games is here to stay and the music executives at Activision and EA have a far harder task ahead of them than composers of original music. Finding the right kind of music to license is incredibly arduous when one has to mix top selling music (usually not thematically related to most games) with a particular game genre. The instrumentation is a key deciding factor in what will please the player and tie them with the events on screen.

Instrumentation adaptability

Finally, we'll take a look at how instrumentation can be varied adaptively with gameplay. In movies the final track is it--there's nothing that can change it. In games you can have the music change each and every session. The first attempt to encourage composers to do this mainstream was in Microsoft's DirectMusic. It gave the composer the opportunity to compose a number of variations and each variation could be a different instrument entirely. It also allowed for a number of compositional changes based on gameplay. What's best is that it was free.

The latest widespread use of this is Creative's Isact--Interactive Spacial Audio Composition Technology. It allows you to layer streams in any speaker setup (from 2 speakers to 5.1 and beyond) and link tracks together easily using either MIDI-controlled instruments or streamed audio tracks. For example, a player could easily introduce behavior to a game where additional tracks could be layered and removed for any definable player activity, like firing a weapon or jumping. The music would all be synced to the correct tempo.

Isact is very impressive, but the hardest challenge isn't necessarily to program automatic events that may or may not hit the player in the right place emotionally or to increase their awareness properly. To incorporate music in a way that makes it feel cinematic and dramatic without overpowering the experience, the real trick is to trigger events that vary for enough moments in a title that makes it truly feel like you're creating your own movie score in a non-linear 3D environment. Very few titles have achieved this, and to be honest, the only one I can think of here is No One Lives Forever. There could actually be a position on a game team reserved for "music integration", where someone spends all their time analyzing events and hooking up the right cues. Such a thing may not be far off.

In conclusion, game music these days is as diverse as any area of the entertainment industry, having pop acts, movie composers, and its own blend of unique and custom-made gems that pop up every once in awhile. Now that the playing field is so vast, we have a great opportunity to truly advance the genre beyond what is being done elsewhere, and instrumentation is one of the most obvious ways to do it.

______________________________________________________