Q&A: Using fear to build an immersive soundscape in Shadow of the Tomb Raider

Shadow of the Tomb Raider, the third entry in Square Enix's gritty reboot of the franchise, exchanged the desolate mountaintops of its predecessor Rise for claustrophobic jungles, bustling hidden cities, and crumbling ruins.

It was a switch-up that took the game's audio in an entirely new direction, with the sound team eager to marry a conventional score with authentic pre-Colombian sound effects to fully immerse players in an exotic world fraught with unfamiliar dangers. 

Intrigued by their mission, we caught up with composer Brian D’Oliveira and audio director Rob Bridgett to learn how they used sound to manipulate player's fears and transport them into their very own heart of darkness.

Brian, you said that you wanted to get 'into the texture' of what darkness is during your three years working on Shadow. How did your understanding of that concept evolve over time? 

Brian D’Oliveira: For both Rob and myself, it was apparent that the score would ultimately end up being a multifaceted tapestry of sounds, with "fear" being the emotional thread that would interweave and tie together the overall sonic fabric of "darkness."

So the first couple of years was mostly spent workshopping and researching together to really identify these core foundational emotions within Lara's world. In tandem with the creative team, we built a musical sound language that ultimately combined in a myriad of ways to become the sound world that we now experience in Shadow of the Tomb Raider.

Very early in the process, it became apparent that the best way to achieve this was to creatively limit the musical palette to only use natural acoustic sources and employ as close as possible the most accurate representations and original instruments. All of the music is performed entirely live, and there is no looping or digital over-editing; it's purposefully imperfect and natural.

You came back from Mexico with eight bags of obscure, exotic instruments. Was it tough to marry together so many contrasting sounds into a cohesive soundscape? 

Brian D’Oliveira: Since I had already spent a lot of time planning and researching during pre-production, I had very carefully selected and even commissioned instruments to be made for me with this in mind. So, once I was back in the studio with them, I had an overwhelming palette to draw from. 

What really helped me make sense of these newfound musical treasures was the second part of my research phase, where I also brought over one of the leading masters of Mexican pre-Colombian music (hailing originally from the Otomi tribe), Ramiro Ramirez Duarte, whom I had met while there. 

We spent over a month living and performing together with these instruments in a show as part of my artist residency at the SAT Montreal dome venue. So not only did I practice and perform this music, I was also able to learn a lot from the audience reactions and got to know the music and instruments at a much deeper level. 

A (rather scary looking) Aztec death whistle

How did you set about recording each instrument, ensuring those strange magic rocks and death whistles (pictured above) you lugged back from South America sounded as authentic as possible?

Brian D’Oliveira: About a year and a half into the production we moved into our new facility that also happens to be the original studio built and used by RCA Victor in the 1940s. The room was designed and built with the concept that it would resonate like a musical instrument […] and you can hear it breathe and react to the notes you play with infinite acoustic variation.

My chief recording and mixing engineer Jera Cravo has a vast amount of experience and knowledge. He was able to effectively combine the best of tried and true techniques along with new tech and workflow hacks, which allowed us to work extremely fast and efficiently. We recorded using multiple stereo microphone sets, combined with UAD unison mic preamps, so depending on our relative location in the room with the microphone we had almost an infinite selection of colors for sound focus, reverberation, and panning position.

Most of the stones and death whistles you hear are non-effected recordings using this approach and feel almost as if they were processed binaurally because of the spatial definition present. Even perceived movements in panning are real physical movements that have been purposefully incorporated as part of the performance.

Rob, as the audio director, how did you translate the musical elements and sounds Brian submitted into the 3D game space?

Rob Bridgett: Musically, Brian would give me the stems for any track or individual piece he sent over. Just incase you aren't aware, stems are specific elements of a musical piece that have been isolated, like only the percussion, violins, strings or FX etc. 

It was in these FX stems that I first got to play around with the death whistles and conch horns, and was able to start playing around with those sounds in the 3D space itself. The idea is always that these sounds are caused by something that is never explained. It is off-screen, in the far or mid-distance (mid-distance is better, because it makes players more uncomfortable). It could be someone or something making those sounds. It could be an architectural design made to instill fear into those who enter that space. 

I like the idea that the player asks these questions of the sounds they hear. What is that? What is causing that sound? In particular, Brian had so many different sounding death whistles, and they offered a really great sound palette for these kinds of architecturally ambiguous sounds.

You've also expressed a desire to move away from the more traditional melodic audio present in previous Tomb Raider titles by blurring the lines between the main score and traditional sound effects. Could you talk us through that process? 

Rob Bridgett: It is all about balance. Emotionally, sound and music are often doing the same thing, giving us cues on how to feel, though traditionally, we expect music to perform this emotional role while sound performs the role of "reality." We decided we can play against those traditions to create the emotional effect we wanted. We place a lot of these musical textures and effects in 3D, and in the space itself to really amplify this sensation of anxiety.

The Mayan and Aztec instruments we used are so unfamiliar to the audience, musically; that they work perfectly in this context, as does our re-use of the original Tomb Raider instrument and the ambient music design Martin Stig Andersen helped us with for underwater sequences. This is all part of the vision to move away from trying to replicate reality with sound, and instead trying to convey a state of mind to the player.

During our mix, we even made the decision to release the music from the shackles of the LR channels and pan it spatially much deeper into the surrounds. The music in the stereo field just felt too safe, like it wasn’t reaching out to the player and into their living room. Again, this ambiguity and anxiety about what causes sounds the player is hearing is the central component of our concept of fear for the soundtrack as a whole. 

Back to the point about balance, that is the whole key to using all these elements effectively. We cannot use this fear technique too much, because, like anything, it becomes tiring and annoying, much in the same way that melody and themes become tiring and overwhelming if used too much. Having a game and a story that is very well shaped in terms of action, traversal, story pacing, darkness, light, fear, hope, helps us a ton in this regard. That is actually one of the cool things about working on a Tomb Raider title, the overall shape of the experience is a joy to work with for sound and music. It is always important that we respect that overall dramatic shape with our approach.

Finally, you've mentioned that you made use of Dolby Atmos. What tangible benefits did the tech bring to the table?

Rob Bridgett: When you think about it, players are always looking through a small rectangular window into our game world, but everything that is around the player off-screen is rendered in sound all the time. The persistence of things on-screen and those off-screen sounds moving behind the player immerses them in this world, as they believe what they're hearing behind them is still there.

Because we knew we’d be using Atmos, about halfway through our development we decided to completely shift away our propagation of sound in the game from 2D, pre-rendered ambiences, and move fully into object-based, 3D prefab sounds for everything like animals, insects, birds, water, rain, dust, debris, and even wind and air tone.

Our level editors author all of that in 3D. It is actually faster to work this way for us, and it gives us a true 3D effect when the player moves the camera, as all those 3D objects translate exactly as you would expect them to. It’s pretty much a VR approach to sound, but done for a none-VR title and on a vast scale. And even though we talk about Atmos a lot, we also support every other 3D spatial audio technology available to us such as Windows Sonic, and Sony’s proprietary Spatial 3D audio platform. If done correctly, what all of these spatial audio technologies bring to the table is the ability to completely wrap the player in your 3D environments. 

Having said that, when pushing immersion in a 'believable" world, take a moment to think about what we said earlier about fear, tombs, and dark jungle spaces all messing with the player's head. Having environmental sound that is all around the player - combined with the fact that what they're hearing may or may not be music, effects or something else they can’t understand from their model of reality - is suddenly a very powerful, and very real fear effect. 

What we're doing is using every trick of immersive "reality," to tell the player something is authentic. It is uncomfortable, possibly too uncomfortable for some people. To me, that is almost the real trick to what Atmos, and other spatial audio technologies bring to games and spatial storytelling in general. This idea of spatial storytelling is extremely fascinating to me. Yes, you can present the same story in mono, but if you present and experience that story spatially, it physically punctures the screen, and is rendered into our real 3D world. This allows our storytelling to enter the 3D space in a meaningful way. 

For an even deeper analysis of Shadow's sound design, be sure to check out the full transcripts from our chats with Brian and Rob right here. Seriously, there's a lot more where this came from.