Strategies in Audio and Music for Portable Games

I'll be talking about effective music composition for mobile and portable gaming platforms during my talk, "From Total War to Assassin's Creed: Music for Mobile Games," which will take place on March 16th at the upcoming Game Developers Conference at the Moscone Center in San Francisco.  With that in mind, I thought I'd use this blog entry to share some resources that explore current strategies and trends in regards to sound and music for mobile - resources that could be useful to the video game composer and sound designer.

While my talk at GDC will focus specifically on music composition and implementation for handheld devices, the resources that will follow in this blog offer assistance with the more general technical issues that face audio pros creating sound assets for a mobile gaming environment.  I've included links to the original articles, as well as a summation of some of the best points that I thought were particularly interesting:

Memory Issues in Mobile Gaming:

Audio For Mobile Games by Jennifer Walden (Post Magazine)

Jennifer Walden is an experienced sound designer and an accomplished journalist.  Her writing focuses on audio post production, and her articles have appeared in such top industry publications as Mix Magazine, Sound & Picture, postPerspective and Post Magazine. In this article from March 2015, she explores the audio design of projects that include FIFA 15 Ultimate Team for Android, Eisenhorn - Xenos, and Wave Champions.  Taking a close look at the challenges posed by each project, Walden discusses the techniques and creative solutions that the audio teams developed along the way.  In so doing, it becomes apparent that memory limitations exert a strong guiding influence on the decision making process of audio pros working in mobile.

In the popular FIFA 15 Ultimate Team for Android game, live recordings of authentic stadium cheering were transferred from the console version of the game to the mobile version. In order to accommodate the memory limitations, the number of recordings had to be limited.  This resulted in a somewhat diminished variety in audience reactions, but ensured that the robust presence of the stadium crowd would still be felt in the mobile version.  Likewise, the voice commentary in the console video game could not be included in the initial installation of the mobile game, since the 400MB size of the commentary far exceeded the 250MB initial download limit.  To address this, the team offered the commentary content to the player after installation as a separate download, which was triggered when the player chose their preferred commentary language (English, French, Spanish or German).

In Eisenhorn - Xenos, the audio team worked hard to maximize the aural impact of this game in the famous Warhammer 40,000 universe, while also economizing on memory by removing some low end from their recordings. They anticipated that the mobile device's speakers would not be able to convey bass with much fidelity.  The audio team also found ways to conserve their memory resources by limiting variation in repeating sounds (such as footsteps or sword swipes), favoring repetition of a small number of sounds over the use of lots of variations.

In the Wave Champions racing game, the team was able to introduce variety in the sounds of collisions by focusing on the materials of the objects involved in the collisions, and then creating a pool of ten variations for each type of material.  For instance, a plastic boat crashing into a concrete obstacle would make a noise consisting of a plastic, concrete, and water sound. Each of these sounds would be selected from a pool of ten variations - for instance, the plastic impact would come from a list of ten plastic impact sounds. Combining this with a concrete impact and a water splash would create a unique sound effect, and these combinations would serve to increase sonic variety during gameplay.

The full article explores the audio issues facing these projects with plenty of useful details and observations.

Loudness Issues in Mobile Gaming:

Adaptive Audio for Mobile by Rob Bridgett (Gamasutra)

Rob Bridgett is a G.A.N.G. award-winning writer and audio developer, currently serving as audio director of Eidos' Montreal studio. In March 2015, while working at Clockwork Fox Studios, he wrote a detailed article for Gamasutra on the audio development of Zorbit's Math Adventure - a game designed to teach kindergarten children the fundamentals of math.  In describing the audio design of this game, Bridgett explains that his primary focus for this project was centered on the issue of loudness in the mobile gaming environment.

Keenly aware of the sensitive nature of children's hearing, Bridgett developed an "adaptive loudness model" that compensated for two factors:

This system attempted to create an optimal volume level by using various traditional compression techniques in tandem with an innovative system of adaptive compression based on the noise level detected by the device's external microphone.  By measuring the "noisiness" of the room in which the game is being played, the adaptive compression system can adjust the perceived loudness of the game to compensate for any competing aural sources.  While some aspects of the system couldn't be practically implemented in Zorbit's Math Adventure and remain theoretical, the possibilities of such a system are intriguing.  Bridgett describes this "adaptive loudness" system in depth within his Gamasutra article, and it's a fascinating read.

Audio Contractors in Mobile Development

Crank It Up!  Social Gaming Needs Great Sound! by Nicholas Day

Nicholas Day is the Creative Director of Plarium Global Ltd., a developer of mobile and social games for hardcore gamers. In May 2015 on the Plarium web site, he wrote an enthusiastic article supporting audio excellence in mobile and social games.  After first making the case for an increased focus on audio quality in these types of projects, Day went on to describe the steps taken for various Plarium projects, including Total Domination and Soldiers Inc.  An important facet of the audio production workflow was the contracting of quality external audio professionals.

While Day acknowledges that audio resources can be created and implemented in-house, he stresses that contracting externally can prove to be more cost effective than acquiring internal personnel and equipment.  Moreover, Day emphasizes that external contractors need not be located within proximity of the developer, and can in fact collaborate effectively from great distances.  "There are quality sound studios throughout the world," Day asserts.  "With the Internet, a global studio is certainly possible."  The article explores the advantages and considerations involved in external contracting for mobile audio, and is well worth reading.

Extreme Memory Issues in Mobile Audio

Raising Hell in Mobile Audio: The Sounds of Dungeon Keeper, by Nick LaMartina (G.A.N.G.)

Nick LaMartina is an experienced sound designer/composer who is currently working as an audio contractor for Magic Leap, Inc.  He previously held an audio director position at EA Mythic, and in January 2015 he wrote a comprehensive article for the Game Audio Network Guild web site about the audio development process of Mythic's Dungeon Keeper game.  In the article, LaMartina emphasizes the artistic ambition of the game's audio development, and the teams' desire to attain AAA audio quality for their mobile game.

However, as the article illustrates, the push to economize on memory resources was persistent throughout the project. "We used every compression format and optimization Wwise makes available," says LaMartina.  He goes on to describe an audio tactic he describes as "a Level-of-Detail system," designed to keep audio complexity under control by deactivating digital signal processing, altering voice limits, pausing submixes and adjusting volume levels as compensation for low-memory situations.  LaMartina includes some interesting technical detail for this system, and for the team's decision to extensively use the Vorbis codec in the game.  The article is a great exploration of the audio workflow of a mobile game with AAA ambitions.

Conclusion

These four articles offers some awesome perspectives on the technical challenges of audio development for mobile platforms, as well as some innovative solutions for the most common problems.  In my upcoming GDC talk, I'll be exploring issues relating specifically to music composition for mobile, which has its own unique challenges.  I'm looking forward to it, and I hope you'll join me in San Francisco on March 16th!  You can find more information about my talk here: "From Total War to Assassin's Creed: Music for Mobile Games."

Winifred Phillips is an award-winning video game music composer whose most recent project is the triple-A first person shooter Homefront: The Revolution.  Her credits include five of the most famous and popular franchises in video gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims.  She is the author of the award-winning bestseller A COMPOSER'S GUIDE TO GAME MUSIC, published by the Massachusetts Institute of Technology Press. As a VR game music expert, she writes frequently on the future of music in virtual reality video games.

Follow her on Twitter @winphillips.