Creating a rich soundscape for the exotic life of Avatar: Frontiers of Pandora.
January 6, 2025
Game Developer Deep Dives are an ongoing series with the goal of shedding light on specific design, art, or technical features within a video game in order to show how seemingly simple, fundamental design decisions aren’t really that simple at all.
Earlier installments cover topics such as cover topics such as how camera effects, sound FX, and VFX created a smooth and high octane movement system in Echo Point Nova, the technical process behind bringing The Cycle: Frontier to Unreal Editor for Fortnite, and how the developers at ROAR Games designed three distinct, overlapping game worlds within one cohesive narrative in Tenet of the Spark.
Hi, I'm Sylvain Livenais, a senior sound designer at Massive Entertainment: A Ubisoft Studio, working on Avatar: Frontiers of Pandora. Avatar: Frontiers of Pandora is a first-person, action-adventure game set in the Western Frontier of Pandora. Players help their character reconnect with their lost heritage and discover what it truly means to be Na'vi.
Pandora is a rich world containing thousands of different plants, animals and insects. I had the opportunity to create the sound for some of these. In this article, we will analyze two specific examples: The Lift Vine and the Veilswarm.
The Lift Vine blurs the line between plant and animal. It’s what the Avatar universe calls “zooplantae” – they may look like exotic plants, but can have surprising, fun interactive behaviors. They might retract when you get close to them (as seen with the “Helicoradian” in the Avatar movie), electrocute you while you’re exploring the world, or in the case of the Lift Vine: pull you up a ledge when you grab onto them.
Some parts of the Lift Vine look like an animal and other parts look like a plant. For example, the tail of the vine looks very much like a reptilian or bird claw, while the top looks like a plant with leaves and branches. We wanted to emphasize both elements by creating organic sound textures, to make it feel grounded and believable, and fit in with the rest of the natural soundscape in the game world. However since they have gameplay functionality, we also made sure they have high transients to make it easily audible for the player, in the busy mix of sounds in the rainforest. The Lift Vine also has several versions with gameplay variations that we will explain later. As it is found very frequently in the game, in every part of the open world, the sound couldn’t be too invasive, repetitive or loud. We wanted the sound to be somewhat appealing and attractive for the player to invite them to use the Lift Vine.
Allows the player to go up or down ledges when they grab the vine.
The vine isn’t available in polluted environment. The player can’t use it.
A second gameplay version of the Lift Vine has the vine inside the bulb (the round part at the top), making it unavailable for the player. They must shoot at the bulb to release the vine.
A third version of the Lift Vine also has the vine hidden, making it unavailable for the player. They must climb and interact with the little plants on top of the ledge to release the vine.
The Lift Vine’s sound is made from a mixture of libraries and original recordings. I recorded various props, such as a leather belt, jacket, deflated balloon, various vegetables, etc. and processed it in Reaper.
Lift Vine in Wwise. Image and caption via Ubisoft.
The source sounds are implemented in Audiokinetic Wwise, using random containers with sound variations. We paid a lot of attention to detail in the implementation. Each part of the Lift Vine was to have its own bespoke, carefully positioned sound, not only when the player uses it but also when it is static in the open world. For example, when the Lift Vine isn’t used, we can see the vine or the claw slightly moving. The leaves in the vegetal platform on top of it are also gently moving according to the current wind strength in the world. All these little things deserved a sound.
Let’s look at all those components in detail, starting with the vine. The vine sound is made of leathery and creaky textures. I remember in the recording room, twisting an old big rope in every possible direction to have as many variations as possible, which was a lot of fun. Two audio emitters are positioned along the vine and the distance from each other varies according to the vine length.
Left, a vine. Middle, 15 meters vine length and right, 30 meters vine length. Image and caption via Ubisoft.
The vine length can be just a few meters long, or up to 30 meters. The audio emitter position changes according to this length so they stick to the vine. This emitter position variation is scripted in the Lift Vine’s graph in Snowdrop, our proprietary game engine. We calculate the real time vine length by subtracting the Y axis float world position of the head to the tail Y axis float world position. Then we divide the difference by a constant. The quotient is then added with the Y axis of the tail world position and the sum is the Y axis of the audio emitter.
These vine creak sounds stop playing when the area in which it is placed, has been polluted by the human antagonists in the game. When it’s polluted, visually, the vine disappears, the bulb and the vegetal platform turn grey and look sickly. We use the audio emitter position and check its pollution value at runtime. When it’s above a certain value, we set a Boolean to false, preventing the audio from playing.
How we dynamically change the audio emitter position based on the vine length in Snowdrop. Image and caption via Ubisoft.
Polluted Lift Vine. Image and caption via Ubisoft.
Snowdrop graph with the pollution value node. Image and caption via Ubisoft.
The claw idle sound, when it’s static and already opened, is made of rope creaks and gravel friction to get a "clicky" organic sound. It gives the claw a sort of animal texture. We didn’t want it to sound too vegetal because it looks very much like a monster claw. So, this part had to sound different than the rest of the plant. Also, as mentioned before, these high transient clicky sounds are made to attract the player’s attention.
Lift Vine claw. Image and caption via Ubisoft.
The upper part of the Lift Vine contains the bulb and the vegetal platform. Sometimes, the bulb hosts the vine inside it (one of the gameplay variations). Visually, when it does, we can see that the bulb is very big and round, inflated. The bulb idle sound remains very organic but emphasizes the fact that it contains something inside. It also plays only when players use their “Na’vi senses” to listen, but we will talk about that later.
Inflated bulb containing the vine inside it. Image and caption via Ubisoft.
In this case, the player must shoot at the bulb to release the vine. We trigger a bulb burst and vine release sound with organic vibes when it ’s destroyed. We wanted the bulb burst to be puffy, grainy, almost smoky, to fit with the big brown seed cloud VFX release. The vine being released needed to sound a bit wet, liquid as we imagined that the inside of the bulb theoretically contained some sort of lifeblood even if we don’t see it. We extrapolated the sound from the visual to give even more life to the plant.
Bulb Explosion. Image and caption via Ubisoft.
Then there is the third gameplay variation: the player must climb and interact with the little plants on top of the Lift Vine to release it. We made a specific one-shot sound for the plant retracting with the same vegetal and organic vibes.
Plant to interact with. Image and caption via Ubisoft.
Image via Ubisoft.
When the player goes up or down when using the Lift Vine, we trigger an “in-ear” wind sound for a better sense of speed. The idea is to mimic something we’ve all experienced in the real world: the low, fluttering sound of air buffeting on our ears when there is a lot of wind or fast movement. It’s one of the ways in which we try to take advantage of the game’s first person perspective, to make the player feel like they’re really there—more immersed in the world and connected to their character. This sound is in 2D, meaning it moves with the player’s position rather than being placed in the world. Therefore we need to manually switch between a stereo and a 4-channel (quad) version, depending on whether the player is using a surround sound system.
In ear sound configuration in Wwise. Image and caption via Ubisoft.
Having the 2D in-ear wind sound emitter attached to the player created a complication. In coop mode, we didn’t want this sound to trigger when the player’s teammate uses the Lift Vine because it wouldn’t make sense for the local player to hear this extreme proximity sound, when a remote player uses the Lift Vine.
Snowdrop node. Image and caption via Ubisoft.
To do that, we set a condition with a Boolean when the agent using the Lift Vine is the local player or a player from the server. This was just one of many considerations that needed to be made to ensure the coop experience works for both players.
As promised earlier, let’s talk about the player character’s “Na’vi Senses”. Basically, the player can go into a “super focused mode” by pressing a button on the controller, and it helps guide them through the game. It highlights important things visually and sonically and enables the player to find relevant information. (In early production, this feature was called Master Hunter Senses, which is why you can see the abbreviation MHS in these screenshots).
Wwise switch container. Image and caption via Ubisoft.
If Na’vi Senses are active, and the bulb is big and round so the vine is inside it and not available, we trigger an extra layer of organic and clicky looping sounds in the bulb. We also do some dynamic mixing with azimuth and elevation RTPCs so it gets more audible when the player is looking directly at it. All these things are very subtle on their own. But when put together, they can guide the player towards the Lift Vine in the dense and busy open world, without becoming overbearing.
Na’vi Senses activated to guide the player. Image and caption via Ubisoft.
Dynamic mixing with Wwise Azimuth and Elevation RTPCs. Image and caption via Ubisoft.
Now we’ve covered all the sounds coming from the Lift Vine, but of course the player is also able to grab onto it, as part of the gameplay functionality. So, we needed some “foley” sounds to cover the characters hands interacting with the zooplantae. To make sure the sound matches what the player sees on screen, we not only had to work on the Lift Vine graph object but also in the player animation system to implement these little sounds.
Wwise implementation for player grabbing the vine sound. Image and caption via Ubisoft.
Audio integration in Snowdrop animation system. Image and caption via Ubisoft.
Working on the Lift Vine was very interesting as it’s a highly interactive feature, and it involved working on a lot of different sounds, such as the sound of the Lift Vine itself but also player foley sounds and 2D in-ear wind components. This also meant learning to work with a lot of different graphs and systems in the game engine and creating solutions to make dynamic spatialization and motion for audio emitters, both in Snowdrop and Wwise. It was super cool to bring all the different parts of this feature together and use sound to help shape it into a cohesive experience for the player.
Image via Ubisoft.
The Veilswarm is a flock of small organisms, that the player’s Ikran (their winged mount and companion) can eat to replenish its stamina. They look like butterflies, but much bigger compared to butterflies on earth, so we wanted to have medium size wing flapping sounds. We didn’t want any kind of tonal screech or bird whistling sound as the Veilswarm is everywhere in the open world and hearing the same type of sound over and over would have become fatiguing. Something a bit more discreet was enough. Also, the open world already has a lot of tonal sounds coming from other flying animals, so there was no need to pile up in this field. Of course, we wanted it to sound like a swarm surrounding the player, and it especially needed to be convincing when the player flies through the swarm and they all disperse. So, we did a lot of work on these flapping sounds in terms of variety and texture, but also in terms of implementation, positioning and movement.
Image via Ubisoft.
If the player is flying on their Ikran and goes through the Veilswarm, the swarm will disperse but the Ikran will catch and eat a few of the animals to gain stamina.
Image via Ubisoft.
The swarm spreads in all directions when the player flies through it. We play a windy whoosh sound when it spreads to give the player a sensation of speed and dynamism. We trigger it according to the player’s speed and distance from the Veilswarm. In addition to the whoosh sound, when it spreads, we trigger a one-shot agitated flapping sound. Another layer is an audio loop in a blend container with various assets based on distance to the player. It helps them find the Veilswarm while flying with the Ikran in the open world, while keeping the sound of the swarm grounded and convincing at various distances.
Image via Ubisoft.
Image via Ubisoft.
If the player stays inside the swarm, while flying steadily, the swarm will surround them. In this case, a 2D stereo or quad agitated Veilswarm flapping sound plays (2D meaning not actually positioned in the world, but all around the player, as with the in-ear wind above). We switch between stereo or quad version according to the player’s speaker configuration. This 2D sound gives a good sense of being surrounded by the swarm by filling the entire sound field with a wide, bespoke audio asset.
When the player moves inside the swarm, the little butterflies will quickly move to the sides, escaping from the Ikran’s attempt to eat them. We dynamically mix the sound using Wwise’s real time parameter controls (RTPC), based on camera movement, player distance from the object, player speed, and azimuth. We tweak the volume, pitch and spatialization using these RTPCs. The flapping sound dynamically increases in volume or pitch, and stays in sync with the visual when they spread on the sides, because we use the same data inputs as the visual in the graph object.
Wwise mixing RTPCs for the swarm spreading sound while inside it. Image and caption via Ubisoft.
For better clarity when in coop, just like with the Lift Vine, we don’t want to hear this 2D sound from the local player perspective if the remote player triggers it.
Wwise switch container for local and remote player. Image and caption via Ubisoft.
The Veilswarm has 2 audio emitters. The first one plays in the origin of the graph object. The other one dynamically changes position according to the player’s position. It mirrors the player’s position, and its axis is the origin of the Veilswarm object. It helps the sound to be more vivid when the player stays inside the swarm and moves around.
When the player moves to the left, the upper emitter moves to the right. Image and caption via Ubisoft.
When the player moves to the right, the upper emitter moves to the left. Image and caption via Ubisoft.
This positional mirroring turned out to be crucial to selling the fantasy of being surrounded by a thousand pairs of flapping wings, and to achieving our goal of making the player feel like the world of Pandora is reacting to them at all times.
Snowdrop audio emitter calculation when mirroring the player position. Image and caption via Ubisoft.
The Veilswarm was also a collaborative work. The player’s Ikran triggers an “eating” animation when going through it and it also involves some VFX when the little animals are being chewed by the Ikran (Pandora is a dangerous place…). The player character may react with a line of dialogue when the Ikran feeds, and the UI stamina bar regenerates with a small animation. So not only did the Veilswarm involve the work of a variety of different sound designers and voice designers, it also enabled us to collaborate with all those other departments working on this part of the experience.
You May Also Like
