Understanding Audio in VR - A Game Music Composer's Resource Guide

By Winifred Phillips | Contact | Follow

When I'm not at work in my studio making music for games, I like to keep up with new developments in the field of interactive entertainment, and I'll often share what I learn here in these articles. Virtual reality is an awesome subject for study for a video game composer, and several of my recent projects have been in the world of VR. Since I'm sure that most of us are curious about what's coming next in virtual reality, I've decided to devote this article to a collection of educational resources. I've made a point of keeping our focus general here, with the intent of understanding the role of audio in VR and the best resources available to audio folks. As a component of the VR soundscape, our music must fit into the entire matrix of aural elements, so we'll spend this article learning about what goes into making expert sound for a virtual reality experience. Let's start with a few articles that discuss methods and techniques for VR audio practitioners.

Methods and techniques

3D Audio formats for VR

3D Sound Labs takes us through the three most popular sound formats for spatial audio in VR: Multi-channel, Object-based, and Ambisonic.

How 3D Spatialized Audio Bottlenecks Virtual Reality Video

VR audio can consume enormous computational resources, resulting in a resource war between audio and video content. This article discusses how audio demands in VR consume memory bus bandwidth, and proposes a short-cut that manipulates the frequency response of the audio content to enhance spatialization while consuming fewer resources.

Adapting Your DAW for VR Audio

A VR sound editor discusses his methods and workflow for virtual reality in this article. Topics include capturing audio in the ambisonic format, and customizing the Pro Tools environment for spatialized audio projects.

Simple spatial audio for beginners

This article summarizes the available entry-level techniques and equipment that can help a newcomer jump into the world of spatial audio for virtual reality.

An audio post production house shares techniques for VR Audio

In this article, we learn about some of the top equipment and methods used by the VR-focused division of an experienced audio post production facility.

Technology and tools

Now that we've garnered some insight about what goes into creating VR audio, let's take a look at some great technology and tools designed specifically for creating virtual reality sound.

The "Works" 3D Audio plugin for Pro Tools

This article explores the 3D rendering technology of the G'Audio Works plugin, which supports multi-channel, object-based and ambisonic spatialization within the Pro Tools application.

The Steam Audio Software Development Kit

This article focuses on the spatialized audio solution for VR developed by the famous Steam software distribution platform. Available as a free download without any royalty requirements, Steam Audio is designed to assist both Unity and Unreal developers in creating and implementing spatialized audio in their projects.

Facebook 360 Spatial Workstation

The Two Big Ears audio company is known for its 3Dception software enabling audio folks to author spatialized audio for VR applications. Now that Facebook has purchased the company, the software has been rebranded as the Facebook 360 Spatial Workstation with added compatibility for 360 videos hosted on the Facebook platform. The software is free for everyone to use in their projects. However, the previous plugin compatibility with Unity, Wwise and FMOD is no longer offered for new users.

Google's Omnitone, the open source project for spatialized sound in VR

This article describes the Omnitone application, developed by Google to combine ambisonic decoding with binaural rendering. Omnitone was designed to deliver spatialized audio for browser-based experiences and apps designed for Android and iOS.

NVIDIA VRWorks Audio

For Windows games and applications, NVIDIA now offers the VRWorks Audio Software Development Kit for implementing spatialized audio in VR for 64 bit Windows apps. For developers working in Unreal Engine 4, the VRWorks Audio game engine plugin can be added directly to the UE4 engine, while future plugins are promised for other game and audio platforms.

The Google VR Audio Software Development Kit

This article runs through the capabilities of the Google VR Audio SDK, which was designed for use in developing applications for Google Daydream and Cardboard. The article includes helpful suggestions for using the audio system to simulate indoor and outdoor environments, advice for creating spatialized sounds that users can easily pinpoint, and tips for ensuring that aural and visual content reinforce each other within the VR space.

Communities & Organizations

Working in virtual reality can be a complicated and intimidating process. Thankfully, there are plenty of online communities and organizations to help us. I've assembled some of these below:

Conferences

Finally, here's a list of the virtual reality conferences, summits and conventions that have featured audio discussions and demonstrations in their past program schedules:

Conclusion

I hope you find the above resources interesting and helpful! If you have any suggestions for additions to this list, please let me know in the comments section below!

Winifred Phillips is an award-winning video game music composer whose most recent project is the triple-A first person shooter Homefront: The Revolution and the Dragon Front virtual reality game for Oculus Rift. Her credits include games in five of the most famous and popular franchises in gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims. She is the author of the award-winning bestseller A COMPOSER'S GUIDE TO GAME MUSIC, published by the MIT Press. As a VR game music expert, she writes frequently on the future of music in virtual reality games.

Follow her on Twitter @winphillips.