Music Composers and Sound Designers in VR: The Headphones Problem

Over the past few months I've taken several opportunities to write about the role that music and sound may play in the virtual reality systems and games that have become famous in the media of late, and which will begin to hit retail during the holidays this year.  Today I encountered a very interesting research paper that warns of a possible problem that may face game developers as they attempt to deliver three-dimensional audio for virtual reality experiences.  I explored some issues regarding three-dimensional game audio in my book, A Composer's Guide to Game Music, and as game composers, I thought that audio for VR might be of particular interest to us.  So, I'm going to share what I learned about the issue from this research paper, including a conclusion which may indicate an imminent problem for some VR gamers.

All of the popular VR systems rely on headphones for audio delivery, but only one (the Oculus Rift) will include built-in headphones as a part of the system.  The rest will allow the consumer to use their own headphones, and even the Oculus Rift allows for its attached headphones to be removed so that the user can replace them with their own “high quality” headphones.

The Oculus Rift, shipping with detachable headphones.

So, here's where things start to get tricky.

What do the words "high quality headphones" mean to the modern gamer?  Well, the gaming website Kotaku held a survey last year so that its hardcore gaming community could vote to determine the very best gaming headphones.  The winner (by a wide margin) was the Astro A50 7.1 Wireless Surround Sound headset, followed by the Logitech G930 Wireless 7.1-Channel Surround Sound headset.  Two surround-sound models had come out on top.  Of the other headsets in the survey, most were stereo rather than surround, and the only other surround-sound headset in the survey was wired rather than wireless.  Clearly, the community had told us what "high quality" meant - and that was a surround sound experience.

The Astro A50 Wireless 7.1 Surround Sound Headset.

Now, here's where we hit upon the problem, and it's explored in the paper "Challenges of the Headphone Mix in Games," written by Aristotel Digenis (lead audio programmer with FreeStyleGames), who presented his paper in February 2015 at the Audio Engineering Society's International Conference on Audio for Games in London.  Virtual reality games will be offering binaural audio to simulate a fully three-dimensional listening environment.  While binaural audio can present an awesome level of immersion and realism, the technology of binaural sound isn't the same as that of surround sound.  In fact, they're fundamentally different.  If gamers have opted to use their own "high quality" surround sound headsets, then they may be experiencing a lower-quality sound environment than the game developers intended.

Many of the highest quality surround sound headphones include the ability to process an incoming non-surround audio signal into a compatible surround-sound mix (essentially imitating surround sound by virtue of some built-in digital signal processing).  If this processing were applied to the binaural soundscape of a virtual reality game, the effect would cause the immersive quality of the audio to deteriorate rather than improve.  Gamers would be left wondering why their stellar top-of-the-line headphones are making their VR game sound lousy.

So far I haven't heard any reps from the three VR system manufacturers address this issue, and gamers should definitely be warned that "high quality" headphones for VR will need to be stereo, rather than surround.  VR enthusiasts who are hoping for the ultimate virtual reality experience may need to purchase some excellent stereo headphones, if they don't already own them.  Without a warning about this issue, some VR gamers may be set up for a nasty sonic surprise.

Winifred Phillips is an award-winning video game music composer whose most recent project is the triple-A first person shooter Homefront: The Revolution.  Her other credits include five of the most famous and popular franchises in video gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims.  She is the author of the award-winning bestseller A COMPOSER'S GUIDE TO GAME MUSIC, published by the Massachusetts Institute of Technology Press.  As a VR game music expert, she writes frequently on the future of music in virtual reality video games.

Follow her on Twitter @winphillips.