What opportunities are there for games that innovate using audio? LucasArts' Jesse Harlin takes a look at what new audio control possibilities technology has brought us, in this article originally published in Game Developer magazine recently.
[What opportunities are there for games that innovate using audio? LucasArts' Jesse Harlin takes a look at what new audio control possibilities technology has brought us, in this article originally published in Game Developer magazine recently.]
The palette for user input within games grows every year. From drum sets to styluses, nunchuks, bongo controllers and beyond, the past few years have seen an explosion of bundled hardware geared toward changing how players interact with virtual worlds.
Many of these technologies are one-off gimmicks. Others, however, have become so ubiquitous as to become either standardized accessories to particular genres or console manufacturer-mandated pieces of hardware.
While gameplay designers continue to utilize these new avenues for player interaction, audio designers are largely stuck in the decades-old mindset that players make noise in-game by pressing buttons on their controllers.
Quite to the contrary, never before in our industry's history have audio designers had more opportunity for creative interaction with players through hardware and software that is readily available and simply waiting for them to utilize.
Sit down with The Legend of Zelda: The Phantom Hourglass and it won't be long before gameplay mandates that you yell at your DS. Surprisingly, this kind of user-created audio input is something that the DS does extremely well, which anyone who has taught their Nintendog to "roll over" or "sit" can attest to.
At the heart of the gameplay mechanic is the DS's internal microphone. Yet, despite being available for use in every DS game by the very nature of its hardware, very few titles incorporate the microphone as a means of interactivity.
Similarly, first person shooters and networked gameplay over PCs and Xbox Live have made headset microphones a must-have for online multiplayer. However, utilizing these nearly-omnipresent audio input devices as part of the single player experience is extremely rare.
Konami's Lifeline
A handful of previous games have flirted with voice recognition and speech interaction, such as Konami's Lifeline and various tactical shooters like SWAT: Global Strike Team, but few AAA games have managed to make player voice interaction a compelling part of their game's soundtrack.
The gameplay potential for this kind of audio interaction is huge, however. Voice print identification and encryption can find its way into espionage games. Imagine calling plays by physically calling plays within sports games.
Consider the gameplay potential in using the range of human speech from whispering to yelling in order to frighten or coerce enemies within platform games like Jak & Daxter or Loco Roco. As audio designers, we can be doing more to encourage user-generated audio input as part of our design.
Over a decade of CD- and DVD-based console development has largely created an environment in which audio designers see their work as relegated to the inflexible confines of a locked disc. However, hard drives and memory devices are now either standard issue or readily available for every gaming platform, though some are more practical than others.
Every PC and PS3 game has access to a built-in hard drive. While PC gamers are used to patches and publishers that push new content into their games, the concept is relatively new to console games. Each major gaming console has its own online store with a variety of demos, games, and additional content such as added levels and extra songs for music games.
The potential for broadening the sound experience of a game after it's been purchased is broader than updated playlists or a handful of new voice lines.
Imagine an adventure game in which the most powerful weapons or treasure were hidden in-game and the only clues to their whereabouts were garbled pirate radio transmissions that could only be purchased and implemented via downloadable content.
Imagine edutainment games for children with continually expandable vocabulary packs or a music game like MTV Music Generator that allowed for uploadable and downloadable user-created collaboration.
The shipped disc isn't the end of the game anymore and audio designers should be considering the gameplay potential of hard drives, storage devices, and online delivery channels.
Speaking of music, with the Xbox 360, Microsoft now mandates that any music occurring during interactive gameplay be replaceable with the user's own local MP3 collection. The text of the actual Microsoft technical certification requirement (TCR) says little more than that. In practice, however, the Custom Soundtrack option has proven to be clunky and artless in its implementation.
This isn't, however, in any way a reflection on the requirement itself. Nothing in the wording of the TCR says that voice ducking or in-game DSP filtering cannot affect the user's music. Theoretically, a game could read the metadata of a user's mp3s and reassign slower ambient music and faster rock or hip-hop tracks accordingly so as to react with an interactive music engine.
Additionally, the Xbox 360 isn't the only platform with the potential for custom soundtracks. PCs, the PS3, and the PSP all contain the necessary hardware to access a ready bank of user-selected music files.
Audio design potential has extended beyond the confines of disc burns and button presses. As audio designers, it's up to us to make sure that our creativity keeps pace.
You May Also Like
