Can Game Music and Sound Combat VR Sickness?

Virtual Reality Sickness: the nightmare of VR developers everywhere.  We all know the symptoms.  Nausea.  Headache.  Sweating. Pallor.  Disorientation. All together, these symptoms are a perfect recipe for disaster. No one wants their game to make players feel like they've been spinning on a demon-possessed merry-go-round.  So, how do we keep this affliction from destroying the brand new, awesome VR industry before it even gets a chance to get off the ground?

In response to this possible VR apocalypse, the top manufacturers have taken big steps to improve their popular devices.  Oculus improved the display on its famous Rift device, Valve introduced a motion-tracking system that helps us orient ourselves and not get nauseous when wearing the Vive, and PlayStation VR incorporated a wider field of view designed to make players feel more comfortable. Even with these efforts, players are still reporting motion sickness symptoms, and the creators of the VR systems have responded by pointing the finger of blame at game developers.  So, if the developers of VR games have to solve the problem, then how can the music and sound folks help? Can game music and sound combat VR sickness?

Let's start with some good news

First, before we explore the potential of music and sound to address the symptoms of VR sickness, let's celebrate one morsel of good news for the game audio community: VR sickness is not our fault (generally speaking)!  According to research, the presence or absence of sound does not make the nausea more likely to occur, nor does the presence or absence of sound worsen the nausea once it has started.

In a 2012 study conducted at Johannes Gutenberg University in Germany, 69 study subjects were shown a presentation specifically designed to turn stomachs.  It turned out that the biggest culprit for motion sickness was the immersive 3D display - when shown in 2D, the video didn't bother people at all, and sound didn't make any difference, whether it was turned on or off.

While there is one example of a very specific sonic element exacerbating VR sickness (more on this subject later), the general presence of sound in a VR experience isn't an issue when it comes to the comfort and happiness of players. So, if (generally speaking) sound isn't a part of the problem, how can it be a part of the solution?

Whistle a happy tune...

At the Toronto Rehabilitation Institute, researchers wondered if music might reduce the symptoms of what they called Visually Induced Motion Sickness (VIMS) commonly experienced in VR environments.  They recruited 93 brave souls to endure a 14 minute presentation of a bicycle ride from hell.  These woozy folks were split into experimental groups, with some groups hearing music during the presentation, while other groups did not. The findings showed that the VIMS was significantly reduced by music, but only if the study subjects found the music to be pleasant.  This description was key.  The music had to be pleasant for the motion sickness to be best reduced.

The idea of using music to combat nausea isn't particularly new.  The Centers for Disease Control and Prevention recommend listening to music as a way to prevent or treat motion sickness without using medications.  Going a step further, a study conducted by Ohio State University in 1998 showed that playing music for patients during high-dose protracted chemotherapy administration resulted in a significant reduction in nausea.  Music is thought to provide a compelling distraction from those queasy physical sensations that might otherwise be felt in full force.

However, the Toronto study is especially interesting because it specifically targeted visually induced motion sickness, as it is experienced in virtual environments and simulators. The effectiveness of pleasant music as a remedy for this specific type of nausea is significant for us as game composers, because it suggests a possible course of action.  Whether we actually follow this course of action is a more complicated issue.

Should we be composing pleasant music for VR?  That's a tough question to answer.  We know that pleasant music can make players feel less queasy.  But pleasant music isn't going to always meld well with a game's environment and activities.

For instance, we wouldn't expect pleasant, happy music during a first person shooter.
On the other hand, first person shooters are among the biggest culprits when it comes to VR sickness.  Owen O'Brien, executive producer of the EVE Valkyrie VR game from CCP games, does a good job of explaining the problem with first person shooters in VR.  "The problem with first-person shooters is that you're running or crouching or jumping in the game but not in the real world, and because it's so realistic it can make some people (not everybody) feel nauseated if they start doing it for extended periods of time."

So, if we can't compose the music to be outright pleasant, perhaps we should think about lightening it up a bit?  At the very least, we can try to adjust our grim, dirge-like musical atmospheres to feel a bit more neutral. And when we have the choice between an agreeable musical score and a dour one, we might want to let our music occupy the sunnier side of the street.

Now on to the bad news

We previously looked at a research study showing that sound, in a general sense, does not contribute to visually induced motion sickness. However, there is one very specific aspect of the aural environment that can exacerbate VR sickness: low frequency sound.

As it relates to noise pollution and unhealthy environments, people have been aware of the low frequency sound problem for a long time.  In 1973, the first global seminar was held in Paris to discuss infrasound (which are sounds that are lower in frequency than the threshold of human hearing). The first international conference on low frequency noise took place in 1980.  In 1982, a paper published in the Journal of Low Frequency Noise and Vibration detailed the problem in depth.  The paper described the ordeals endured by clerical workers whose offices were unfortunately in close proximity to a testing site for aircraft engines. The low-pitched humming soon caused fatigue, headaches, nausea, disorientation... all the classic symptoms of motion sickness.  These symptoms also showed up in lots of other contrasting situations, including:

All these miserable people shared the common experience of woozy nausea induced by their aural environment.  This effect, however, was brought about by a specific cause: an ever-present, consistent humming or throbbing in the low frequency range. In the aforementioned paper from the Journal of Low Frequency Noise and Vibration, the offending frequency range was defined as any frequencies lower than 261 Hz.

So, do we completely avoid any low frequency sound below this threshold? Well, that would certainly make for a depressingly treble-dominant mix. Sound effects would have no whoomph, music would have no bass, and most gamers and game audio pros would be very sad. However, when composing music or designing sound for VR, we can try to avoid consistent, ever-present, throbbing sounds in this frequency range. For instance, in a VR game set inside a space station, perhaps it might not be absolutely essential to hear the low thrum of the engines and ventilation systems all the time.  Likewise, a musical score for a VR game may want to avoid using a consistently low and sustained bass note over an extended period.

The paper from the Journal of Low Frequency Noise and Vibration also offers an alternative solution: masking the low noise with a simultaneous high one.  "It is not unreasonable to expect that a higher frequency noise of even moderate level may effectively mask either low frequency or infrasonic noise," the paper suggests. "Practical experience shows that this is indeed so." So, when it is possible to finely control the mix and ensure that high frequencies are consistently masking lower ones, then we can include those low sounds without worrying about their queasy side effects.

Some interesting theories to consider

We've taken a look at some well-documented effects that music and sound can have on players, and how those relate to the issue of VR sickness.  Now, just for fun, let's take a look at a few miracle devices that were designed to be motion sickness cures:

The VR Sickness Device

In 1999, inventor Bruce Kania filed for a patent described as an "Apparatus and method for relieving motion sickness."  The device was described in the patent as an invention "to relieve motion sickness which may occur during video games including virtual reality games."  In theory, the device would have included a sensor that could detect motion and then deliver "sensory signals" to the user that corresponded with that movement.  Theoretically these signals could have served to resolve the conflict between visually perceived movement and the corresponding motion detected physically by the user, thus preventing motion sickness. If this patent had been realized as an actual device, then our imaginations can conjure up some sort of built-in component in all VR headsets, delivering these sensory signals and solving the VR sickness problem once and for all.

How does this relate to game audio?  Because some of the sensory signals would have been aural.  The patent describes these signals as "white noise signals, pink noise signals, brown noise signals, and audio tone signals."  Would these audio signals have interfered with the sounds of a game, or could they have been subliminally buried inside a mix?  Would they even have worked?  Who knows. The device was never actually produced.  It's an intriguing idea, though.  This brings us to another technology that promises to cure motion sickness ... and this one is actually available to the public:

You're never sick when you use Nevasic

In a 2003 study conducted by several psychology professors from the University of Westminster and the Imperial College School of Medicine in London, 24 volunteers were placed on a rotating turntable and spun around like tops. The control group were left to twirl around and turn green with no additional help.  The rest of the subjects were split into two groups, and these suffering souls were given one of two remedies. Half were told to focus on their breathing.  The other half listened to a special music recording.  According to the research paper that was published in the Journal of Travel Medicine, "subjects were played a commercially available music audiotape (Travelwell) of “specific frequencies and rhythms blended with orchestral music,” which are claimed by the manufacturers to reduce motion sickness."

The study found that the Travelwell music helped the musically-accompanied spinners to not feel motion sick for a longer period of time than those who had been concentrating on their breathing, or who had just been doing nothing while helplessly whirling around.  So, is this Travelwell music the answer?  And what exactly is it doing that's alleviating the queasiness?

Well, we can hear it for ourselves, because now the Travelwell music is available as an app called Nevasic.  The makers of Nevasic proclaim on their website, "We have demonstrated that issuing the specifically identified and constructed tones, frequencies and pulses in this programme to the ear in a direct mode we disrupt the normal signal chain at the Vestibular level and therefore affect the chain. By affecting the chain in this manner we have also demonstrated the ability to stop and prevent emesis or sickness."

This solution, as described on the web site, seems to have elements in common with Bruce Kania's proposed device for relieving motion sickness.  Both approaches refer to audio signals that affect the inner ear and alleviate the conflict between actual and perceived motion.  So what are the audio signals that Nevasic uses?  We don't know.  No information about this is offered on the company web site or within the application itself.

In the interest of science, I purchased the Nevasic app and listened to the 27 minutes and 10 seconds of audio content that is designed to relieve and prevent motion sickness.  Here's a brief description of what I heard:

The recording begins with a series of bell-like tones. Then waves crash against a shoreline. After a while, a walking synthetic bass takes over, accompanied by some synth chords, high-pitched mallet accents and a snare/kick drum combo. This goes on for some time, and then the ocean waves return. The waves recede in favor of another synth-driven musical groove with a bell-like melody. By the time the Nevasic program reaches 12 and a half minutes, the music has introduced some kind of synthetic drum sound in a high pitch with a randomized rhythm that is alternatively hard panned to stereo right or stereo left. This continues for quite a while. Finally, when we've listened to 27 minutes and 15 seconds of the Nevasic program, the rhythmic elements disappear and we again hear the bell-like warbling tones from the beginning of the recording, which fade to silence once we reach the full 27 minutes and 10 seconds of the Nevasic audio program.

From my listening experience, I couldn't identify the "specifically identified and constructed tones, frequencies and pulses" that were being used, though it seemed clear that pink noise was being delivered via the ocean waves (and perhaps the snare drums as well).  The hard panning and stereo effects seemed deliberate, and I imagine that the randomized quality of these sonic events was designed according to a plan... but I couldn't discern it, and Nevasic isn't offering any information through their site.  If the Nevasic folks have the answer to the motion sickness problem, they aren't sharing their secrets. That's a shame, because if the solution to VR sickness is a special cocktail of audio signals, then we game audio pros would definitely want to know about it.

Conclusion

In this article I've tried to gather together some useful thoughts about the relationship between VR sickness and game music/sound.  Some of the concepts in this article are practical and can be applied to our projects, while others are strictly theoretical.  However, all of these ideas outline the significance of audio in attaining an honest-to-goodness VR sickness solution.  Game audio has a role to play, and I hope we'll continue experimenting and thinking about how game audio can improve the experience of our VR gamers, and keep their stomachs feeling happy and strong.  Thanks for reading!  If you've heard any ideas that relate to this blog, please feel free to share them below in the comments!

Winifred Phillips is an award-winning video game music composer whose most recent project is the triple-A first person shooter Homefront: The Revolution.  Her credits include five of the most famous and popular franchises in video gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims.  She is the author of the award-winning bestseller A COMPOSER'S GUIDE TO GAME MUSIC, published by the Massachusetts Institute of Technology Press. As a VR game music expert, she writes frequently on the future of music in virtual reality video games.

Follow her on Twitter @winphillips.