Editorial: Why Mobile Games Need Better Audio

In the almost ten years of mobile games, advances in technology have moved them from 2D black and white arcade-style classics to full-color 3D role-playing epics. But while graphic capabilities, game play and processing speed in mobile gaming have jumped leaps and bounds, mobile game audio has fallen behind and needs major modernization.

Mobile game audio began almost ten years ago with games like Nokia's Snake. These games featured soundtracks similar to the ringtones of the time-single, dull bleeps at different pitches. Over time, the single bleeps evolved into polyphonic MIDI, giving game developers a palette of actual musical instruments for composing soundtracks. This change made the soundtracks more listenable but left out much to be desired in terms of sound effects realism. The sounds of an explosion, for example, had to be synthesized from a combination of musical instrument sounds. Not very realistic.

Major Advance to WAV & MP3 Formats

The next major advance was the support of sampled sounds in formats like WAV, then MP3. Being able to include sampled sounds meant that game audio could include sound effects designed specifically for that game-finally an explosion sounded like an explosion. However, these improved sound effects came with a trade-off-significantly increased file sizes. And since most platforms don't support simultaneous layering of audio tracks, the developer or the user had to choose between getting sound effects that matched the action on screen or a musical soundtrack. Neither choice provides a complete audio experience.

In recent years, mobile game audio technology has made two long-needed advances to overcome these problems. Some phones now support simultaneous playback of multiple sounds. So mobile gamers finally can listen to the musical soundtrack and sound effects of a game at the same time, rather than having to choose one or the other-a long overdue development considering this has been the norm in console video games for almost 25 years.

Mobile DLS/XMF Reduces File Sizes

Another concept of console game audio was introduced to mobile games to address the problem of the increased file sizes of sampled sound effects-mobile DLS and XMF-audio formats that achieve the same realism as sampled sounds without such large files.

Mobile technology companies like Faith West Inc., Beatnik and Nokia have been among the first to realize the potential of these formats and offer tools used by game developers today to create these powerful formats. In fact, Faith West believes so strongly in mobile DLS/XMF for game audio that it offers developers a downloadable library of 125 game sound effects in that format free of charge.

Support for multiple simultaneous sounds, as well as support for mobile DLS/XMF, has yet to be widely adopted by phone makers. But many game developers have been quick to take advantage of them on phones where they're supported. And games that use these technologies stand out among mobile games because their rich, realistic audio makes them much closer than others to full-fledged video games.

Game Audio Remains at 1988 Level

As more and more phones support simultaneous sound effects and background music, as mobile DLS/XMF support increases, and as game developers continue to take advantage of these capabilities, mobile game audio will gain ground on its graphics counterpart. But despite these advances, most mobile game audio has reached the equivalent of console games circa 1988, while graphics continue near 1996 equivalents.

Fortunately, several recent advances in audio are beginning to reach mobile phones, beginning, as usual, in East Asia. Sound processing, such as reverb and chorus, helps simulate cave and underwater environments, for example. And an effect known as stereo widening can make certain sounds seem farther from the player than others, expanding the perceived game environment.

3D Positioning Most Exciting Advance Yet

The most recent, and perhaps most exciting, development in mobile game audio is the advent of 3D positioning, which gives developers control of where a sound comes from in a completely 3D environment. Using 3D positioning, the sound of an airplane exploding behind the user can come from behind and above the player's head, helping complete the perception of an 3D world created along with 3D graphics. Phones supporting 3D audio are expected to arrive in the U.S. and other markets this year.

Beyond these current developments, the hot topic of next-generation game audio, including mobile game audio, is adaptive soundtracks. Adaptive soundtracks promise more seamless transitions in game music based on what's happening on screen. The concept is that game music will in a sense "compose itself" according to game play. However, adaptive audio is still in its early stages and won't be fully realized for some time.

So why does mobile game audio remain so far behind? The most common answer is that people don't believe game players care about audio because they usually play in public, so they play with the volume off. While this may be true some of the time, it's certainly not true all the time. Conversely, many game players value a game's audio so much that it's common to see them playing PSPs and Game Boys using headphones. Maybe they're not doing the same with mobile game audio because there's nothing worth hearing. Game developers and phone technology makers that address this void early will no doubt benefit from pioneering in a long neglected area.

[Mike Curtes has been working with mobile multimedia for over four years, beginning in Tokyo at Japan's largest mobile entertainment provider, Dwango. He is currently technical project manager at San Francisco-based Faith West, where his work includes supporting the integration of the company's wavetable and synthesizer into Qualcomm's CMX^TM multimedia solution, analyzing emerging technologies, and working with content and service developers.]