Tips, Tools and Guidance: VR Assets for the Game Music Composer

By Winifred Phillips | Contact | Follow

Welcome!  I'm videogame composer Winifred Phillips, and my projects lately have included music composition for a lot of great virtual reality games.  It's been fascinating work!  Last year, when my VR work started to really pick up, I wrote an article with lots of resources to help video game music composers become more comfortable in the world of VR audio development.  Since this discipline progresses rapidly, I thought it would be best to post an update article now that adds additional resources to address new developments in the field.

VR development is continuously innovative and cutting-edge, and I've been fortunately to experience this first-hand.  As an example: one of my more recent virtual reality game projects was music for Audioshield Fitness, developed by the creator of the famous Audiosurf music-rhythm game. I was asked to compose the new official Audioshield Theme for release with the Audioshield Fitness game, which takes the core game mechanics of Audioshield and pumps up the challenge with obstacles that make players dodge and duck to the music.  The result is an intense workout that was named as one of the top 5 VR Fitness Games of 2018 by PerfectBodyMate.com. To maximize the power of the Audioshield procedural system, my composition had to attune itself to the system’s powerful music analysis algorithm and deliver moments of both challenge and spectacle. I composed and mixed the music with specifically-targeted EQ frequency ranges where I placed rhythmic elements and punchy crescendoes.  The Audioshield music analysis system then reacted to this audio content and changed the pacing and content of gameplay to match these variables.  It was a fun challenge!  Here's a video showing how that worked:

Composing for virtual reality is its own unique discipline, requiring a specialized set of skills and tools.  In this article, let's collect some resources that explore the techniques, tools, and technologies associated with VR audio development.  Let's also take a look at the professional community of VR developers that are there to help each other through the rough spots.  Ready?  Let's go!

Technology and tools

First, let's consider some of the tools and technology that has been designed to help audio folks create awesome virtual reality sound, starting first with new innovations over the past year:

Google Releases Resonance Audio

Google has entered the VR business in a big way, first with Google Cardboard, then with the Google Daydream device.  Now, Google has released its own multiplatform software development kit for VR audio.  This article sums up some of the features.  

Below, I've also included a video demonstration of a binaural and ambisonic demo for Resonance Audio:

Let's Test: 3D Audio Spatialization Plugins

In this article from the Designing Sound site, three spatializer plugins are tested for their performance in localizing sound sources under an assortment of conditions: circular movement, acoustic shadowing, vertical position, occlusion, moving occlusion, modeled attenuation, and near-field effect. Similarities and differences between the plugins are discussed, and multiple audio samples are provided for comparison.

MixOnline: The dearVR 3D Audio Reality Engine

This system from the Dear Reality company combines a plugin suitable for integration with Digital Audio Workstations, and a virtual reality application that allows users to instantly test their DAW mixes within a VR environment.  The MixOnline article (linked above) argues that such a system is long overdue and necessary for effective VR audio production workflows.  Below I've also included the demo video for this technology, produced by Dear Reality:

Envelop for Ableton Live

This suite of spatial sound tools is designed to integrate into the Ableton Live Digital Audio Workstation.  For Ableton Live users, the Envelop suite of tools allows effective mixing for a virtual space.  The article (linked above) discusses many practical applications for this software, including virtual reality.

Below, I'm including the links that were offered in last year's article, since the information is still relevant:

The "Works" 3D Audio plugin for Pro Tools

This article explores the 3D rendering technology of the G'Audio Works plugin, which supports multi-channel, object-based and ambisonic spatialization within the Pro Tools application.

The Steam Audio Software Development Kit

This article focuses on the spatialized audio solution for VR developed by the famous Steam software distribution platform. Available as a free download without any royalty requirements, Steam Audio is designed to assist both Unity and Unreal developers in creating and implementing spatialized audio in their projects.

Facebook 360 Spatial Workstation

The Two Big Ears audio company is known for its 3Dception software enabling audio folks to author spatialized audio for VR applications. Now that Facebook has purchased the company, the software has been rebranded as the Facebook 360 Spatial Workstation with added compatibility for 360 videos hosted on the Facebook platform. The software is free for everyone to use in their projects. However, the previous plugin compatibility with Unity, Wwise and FMOD is no longer offered for new users.

Google's Omnitone, the open source project for spatialized sound in VR

This article describes the Omnitone application, developed by Google to combine ambisonic decoding with binaural rendering. Omnitone was designed to deliver spatialized audio for browser-based experiences and apps designed for Android and iOS.

NVIDIA VRWorks Audio

For Windows games and applications, NVIDIA now offers the VRWorks Audio Software Development Kit for implementing spatialized audio in VR for 64 bit Windows apps. For developers working in Unreal Engine 4, the VRWorks Audio game engine plugin can be added directly to the UE4 engine, while future plugins are promised for other game and audio platforms.

Methods and techniques

Now, let's check out some recent articles that offer methods and techniques for creating immersive virtual reality sound:

How to Build Audio for VR Games

In this article, the audio director of CCP Games delves into how human beings perceive sound, and how that extends to VR.  Included: the virtues of hard panning stereo in VR, and the importance of adaptive music in VR.

Oculus' Audio SDK: Localization and the Human Auditory System

While this article is primarily a component of the Oculus Audio SDK documentation for VR audio development, it includes a concise and thorough breakdown of the components of sound that enable human beings to localize sound sources.

Immersive Audio for VR Workflow

This is a video recorded at the VR LA conference, in which a panel of game audio experts discusses a variety of topics connected to VR audio development.  Included: an exploration of the ways in which audio pros can lobby for the importance of audio content within a VR development team, and what features would be most welcome in audio production tools to make them more useful in VR development.  Watch here:

Audio Design for Interactive Narrative VR Experiences

In this audio postmortem of the VR project The Price of Freedom, the audio director explores how to prioritize audio implementation tasks during development, and how to effectively place audio elements within VR environments.  Mixing in VR is also discussed.

9 Things You Should Know When Creating Sound for Virtual Reality

This article published by the GameSoundCon organization lays out several issues to consider when creating and implementing sound content in VR titles.  Among the topics addressed: equalization issues pertaining to HRTF processing, and natural human barriers to sound localization that can be addressed with careful spatialization choices.

As before, I'm including the links that were offered in last year's article on this topic below, since the articles share a wealth of professional solutions to VR audio problems, and their insights can still be useful:

3D Audio Formats for VR

3D Sound Labs takes us through the three most popular sound formats for spatial audio in VR: Multi-channel, Object-based, and Ambisonic.

How 3D Spatialized Audio Bottlenecks Virtual Reality Video

VR audio can consume enormous computational resources, resulting in a resource war between audio and video content. This article discusses how audio demands in VR consume memory bus bandwidth, and proposes a short-cut that manipulates the frequency response of the audio content to enhance spatialization while consuming fewer resources.

Adapting Your DAW for VR Audio

A VR sound editor discusses his methods and workflow for virtual reality in this article. Topics include capturing audio in the ambisonic format, and customizing the Pro Tools environment for spatialized audio projects.

Simple Spatial Audio for Beginners

This article summarizes the available entry-level techniques and equipment that can help a newcomer jump into the world of spatial audio for virtual reality.

An audio post production house shares techniques for VR Audio

In this article, we learn about some of the top equipment and methods used by the VR-focused division of an experienced audio post production facility.

Communities & Organizations

Virtual reality audio development can be daunting.  Fortunately, there are lots of online communities and organizations that can provide advice and encouragement when needed.  I've assembled links to some of these below:

Here are the communities and organizations that were included in the article from last year:

Conferences

Finally, here's a list of the virtual reality conferences, summits and conventions that have featured audio discussions and demonstrations in their past program schedules:

Here are the conferences that were included in the article from last year:

Conclusion

I hope you find the above resources interesting and helpful! If you have any suggestions for additions to this list, please let me know in the comments section below!

Popular music from composer Winifred Phillips' award-winning Assassin's Creed Liberation score will be performed live by a top 80-piece orchestra and choir as part of the Assassin's Creed Symphony World Tour, which kicks off in 2019 with its Los Angeles premiere at the famous Dolby Theatre. As an accomplished video game composer, Phillips is best known for composing music for games in five of the most famous and popular franchises in gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims.  Phillips' other notable projects include the triple-A first person shooter Homefront: The Revolution, and numerous virtual reality games, including Scraper: First Strike, Dragon Front, and many more.   She is the author of the award-winning bestseller A COMPOSER'S GUIDE TO GAME MUSIC, published by the MIT Press. As a VR game music expert, she writes frequently on the future of music in virtual reality games.

Follow her on Twitter @winphillips.