How did one sound designer create the expressive audio canvas which underlies Thatgamecompany's acclaimed indie title? Sony Santa Monica's Steve Johnson explains -- and presents isolated samples so you can listen to his work.
October 10, 2012
Author: by Steve Johnson
'Sup, internet. I'm writing to tell you a bit about what went into the sounds of a beautiful little game called Journey. I was the sound designer on the project, working with ThatGameCompany part-time start-to-finish for several years from my office at SCEA Santa Monica, just a few blocks away from them. People seem to really love the game, and I figured there are too many stories in these sounds for the world to never know, so I thought I'd share them with you in this archive of industry love, Gamasutra.
Beware, though: spoilers lurk below
The singing in the game is of four types: a light quick button press for a "coo", a hard quick press for a "chirp", a reasonably-held press for a "call", and a long-held press for a large "shout".
They are a combination of re-pitched and processed birds, along with musical elements provided by composer Austin Wintory himself. Austin made musical parts for each type to compliment the score in each of the levels, and when close to another player, there are falling variations from your character, and rising variations from your companion.
The buried vocal is Lisbeth Scott, the singer of the final credits track, who is also the main element of the angelic white ancestor figures.
Singing in Barrens level by Journey Sound Design
We considered mapping different calls to different buttons or the D-pad, and even a sort of wah-wah to the right analog stick, but one button was just better -- like the simplicity of the one button in Shadow of the Colossus. The call mechanic is more context- and distance-based in that game, and we wanted to give more control for co-op expression, but I'd still consider Shadow of the Colossus very much an inspiration.
Most of the sand sounds were recorded in my room at Sony, in a low 3' x 3' cardboard box filled with dirty sand I got from Venice Beach. The sand waterfalls in the game were made by pouring the sand from a couple feet high into the box, filling my room with dust. (Are there long-term health effects?)
There are large sand waterfalls made from a lot of sand poured from a recycling bin, and there are small, thin versions made by pouring sand through a poster tube. For the large ones there is a washed-out distant perspective layer and a brighter, rumblier layer when you get close.
Sand Waterfalls by Journey Sound Design
The walking and running sand steps were a combination of jabbing the sand with my fingers, cloth sounds, and recordings made on an early research trip to Pismo Beach with Thatgamecompany. The steep sand climbing footsteps are mostly the socked feet of art lead Matt Nava on a steep Pismo dune. Left steps were distinguished from right steps for a back-and-forth sound.
Sand Footsteps - Normal and Steep by Journey Sound Design
The footsteps on stone came from a hacky sack that sits on TGC designer Nick Clark's desk. The footsteps on metal were made by tapping the side panel of a PC case held loosely against the grated shelf of a mini-fridge.
Stone and Metal Footsteps by Journey Sound Design
All of the player's main robe movement sounds are from a military jacket that got me through Chicago winters -- a gift from a girlfriend. The standing-up-in-sand sound was that jacket half-buried in sand, lifted out of the sandbox. The sounds of walking through a sand waterfall are scoops of sand tossed at the jacket. For standing under a waterfall, it's the sound of sand poured on it.
Standing Up - Walking Into Sand Waterfall - Sand On Player by Journey Sound Design
The sounds of the rolling sand waves have short and tall versions, and were made by putting a headset mic in a Ziploc bag, burying it in sand, and sweeping the sand over it with my hand. There are also pitched-down spinner fireworks that play as sweeteners.
Short and Tall Rolling Sand Waves by Journey Sound Design
The sandbox wasn't big enough for getting the big sand sprays, so I recorded them at 1 am in the playground of nearby Clover Park -- and ducked from cops!
The player's scarf is made from a microfiber apron that was sweet-talked out of a salon by Phil Kovats, my former manager and the audio lead of The Last Of Us. There is a loop that plays from the middle of the scarf, and a loop that plays from the end of it.
The robe and scarf loops are always reading a few variables from code in order to play at the appropriate speed and intensity. They take into account player velocity, wind speed, and for the scarf, scarf length. So each scarf loop is always playing one of 64 specific arrangements of scarf sounds.
Scarf - Half Length Walking Light Wind by Journey Sound Design
The flapping of the giant cloth banners came from a heavy, ugly denim duvet cover I got when I was 13.
Large Cloth Flaps Light Wind by Journey Sound Design
The sand surf sounds were actually repitched, EQed, grass loops from Flower (plus separate loops for turning/carving the sand)... until the day before everything wrapped. I was the only one who knew or cared, but it always bugged me, so I recorded and replaced it with what's there now, what I always thought it should be, my finger drawing circles in the sand. Took about five minutes to do!
Sand Surf and Turning Sweetener by Journey Sound Design
I ruined a pair of old jeans when rubbing them across the parking lot for the sound of surfing across stone, but it was worth it.
Stone Surf by Journey Sound Design
The sound of gliding along cloth bridges is a spoon on fabric.
Bridge Glide by Journey Sound Design
The chime-like sounds that play from the tombstoney things and in the menus are actually re-pitched drill bits being struck.
Tombstone Symbols Turn On - Fly Out by Journey Sound Design
Most of the shining magical stuff was made by mutating and convolving various bells and chimes against each other using a free program called Soundhack.
Scarf Collect - Absorb - Swirl - Grow - Fill In by Journey Sound Design
The bridge stitching sound in the game is the shining magical stuff mixed with my fingernails raking the carpet of my room.
Cloth Bridge Stitch by Journey Sound Design
The sound of the large cloth banners getting colored in was made from the shining bell and chime stuff along with soft liquid lapping, to contrast the dry desert. They pitch up according to how colored in they are, like the flight meter of the scarf. (This is done in code, and not heard in the example below.)
Cloth Color Ins and Completion - No Pitching by Journey Sound Design
The lantern hums in the game were made from meditation crystal singing bowls. Their lighting up sound includes a bell that sits in my parents' living room.
Lanterns by Journey Sound Design
The splash sound of diving in and out of the spirit water in the vertical 6th level came from a hall in Naughty Dog, when Damian Kastbauer, who was there helping with sound implementation on Uncharted 3, walked up behind me and swirled real life sparkles around my head -- a tiny plastic box with a thumb wheel on it that played "Pop Goes the Weasel" at any speed. The little guts of a music box or baby crib mobile, maybe? Timestretched, EQed, effected bursts of it became the magical splashes I needed.
Into and Out Of Spirit Water by Journey Sound Design
Austin also provided musical flourishes that were combined with chimes for collecting the flying cloth strand bits. Collect one at a time and it plays individual small parts; sing to five or more at once and it instead plays a moderately sized flourish; sing to 10 or more at once for a large one. So that way it wasn't just a large stack of individual sounds, but something that sounds like a section of the orchestra springing out of the score.
Cloth Strand Pick Ups by Journey Sound Design
The schools of tiny flying cloth strands were made from processed bats and pitched up cloth flaps that programmer Martin Middleton was able to trigger from every tiny change in direction. For every cloth strand you see there are always multiple little grains of sound trying to play.
Cloth Strands Flying by Journey Sound Design
The calls of the friendly cloth fish creatures introduced in the third level are pitched down and processed aviary birds meant to sound kind of like dolphins. At first they played all of their sounds at random and seemed pretty dumb and robotic.
So I divided them into 16 categories: short, medium, and long idle chatter, mutters, laughter, questions, waking up, responding to a player, "come here", "uh-oh", "found something!", "I'll pick you up", "there you go", trapped cries of help, freed celebrations, and warbled frozen whimpers.
Designer Bryan Singh painstakingly added them in across the game as needed through a combination of script and hard code, and suddenly they seemed halfway intelligent. The jellyfish sounds are related but lower and were meant to sound like they’re getting tickled.
Assorted Fish Calls and Jellyfish Reactions by Journey Sound Design
The giant cloth creatures' happy calls and the flying robots' sad eerie moans were made from humpback whales. According to my friend Sara Kerosky at Scripps Institution of Oceanography, humpbacks have a wide range of sounds, and each population sings the same song, which can last up to several hours. The song is picked up by other populations so that this one song (or calling sequence, or pattern) is sung by humpback whales worldwide each mating season, and it changes slightly each year. They surface to breathe at certain points in the song, never missing a beat.
Giant Cloth Whale Calls and War Machine Idling Vox by Journey Sound Design
The large flying enemies, which we mostly came to refer to as "guardians" or "war machines", were meant to be ancient but advanced stone robots. At the end of the opening ascent of the last level, the player originally used to shout and break apart their stone shells with a giant singing hawk shout, revealing cloth creatures inside. The wandering guardian / war machine moans are processed humpback whale songs and leopards run through a Kaoss Pad named K.I.T.T. for glitchy treatment, as are the digital babbles that play from the broken random body pieces you free cloth from in the earlier levels.
Their attack vocalizations were made from various vocoded animal roars, each with near and distant versions. Those emit from their head, and from three points on their body they emit power throb sounds made from bass drones gated with a square wave. The visor lowering and opening sounds include a giant, old, and very dangerous office paper cutter that was about to be donated to charity. (Nope, still mine!) Their searchlight emits a sound made from the static hums of TV tubes. Their eating cloth sound is made from vacuums and power saws, and the scarf getting ripped has dry ice-on-metal squeals and a gunshot.
Assorted War Machine Sounds by Journey Sound Design
All of the levels have multiple streaming surround ambiences made from library sounds and original recordings. In the snowstorm mountain level alone there are 16 different ambiences for the various areas and wind intensities.
Various Snowy Mountain Level Ambiences by Journey Sound Design
What is the sound of a super hot, very still desert? I don't know, but the ambience in the third level is made from processed hissy warehouse room tone.
Desert level Ambience by Journey Sound Design
The stone pillars the player hides behind in the snowy mountain level originally weren't slated for sound but looked like giant resonators. Their sounds were made by blowing on some bottles, mostly a large champagne bottle -- whose contents were consumed, for the good of the game, on Thanksgiving with a girlfriend. I filled them with various amounts of water for different specific notes, tuning them to the key of the music with my phone's guitar tuner app.
Whistle Stones by Journey Sound Design
In the snowstorm mountain level, the player and guardian sounds buffet in and out with the wind by subtracting the values of randomized LFOs from their volumes, and scaling that reduction according to the wind speed.
The snowstorm mountain level is based around a coldness mechanic. The player's singing grows more randomly detuned the colder and weaker the player gets.
At the start of the level the snow is wetter, and the footsteps were made with the help of my niece and nephews (Isabel, Wyatt, and Jameson!) one winter holiday. Later in the level the snow is colder and drier, and footsteps were made with the old foley standby, cornstarch.
Snow Footsteps by Journey Sound Design
The player's frozen cloth movement also grows with the cold, and the sounds were mostly made from the snacks in Sony Santa Monica's kitchens. It builds from Cheetos, to Lay's, to Sun Chips, to Funyuns, and a deadly cocktail of all for the final death walk. Also, I soaked paper towels and the aforementioned ruined jeans in water, froze them in the freezer, and recorded the sound of them bending. (Apologies to the colleague who found brown ice on his or her Lean Cuisine -- I should've washed the jeans first.)
Frozen Player Sweeteners by Journey Sound Design
The crackle sounds of cloth freezing in the mountain level were made from ice dropped into warm water after being deep frozen with dry ice. The sounds of cloth melting were made from girls sighing.
Cloth Freezes by Journey Sound Design
The player's flight wind is a modified wind system from Flower. The main billowy bassy sound is my open moon roof while going too fast on a San Diego freeway. There is also another loop for altitude that pitches from a high thin airstream up high in the sky, down to low breathy air rush near the ground. Their volumes scale with player velocity.
The glowing ambience at the very peak of the mountain, and buried in thewhite ancestor scenes, is mostly a favorite song of mine that's been smeared into a heavenly drone. I stretched it out like 1000 percent using a free program called Paul's Extreme Sound Stretch, then I reversed, layered, panned it, and threw in some sparkles. This is my best attempt at the sound of heaven.
Ancestor Heaven Ambience by Journey Sound Design
The misty waterfalls in the last level that look as if they're turning into snow midair were made from an effected version of me saying, "Sssssssssssss". The sound that plays when you pass through a waterfall is the spray from a spray bottle.
Snowy Thin Summit Waterfalls and Passing Through by Journey Sound Design
The effervescent beacon the player has to sit in at the end of each level has champagne, Alka-Seltzer in water, and baking soda in vinegar.
Ancestor Beacon by Journey Sound Design
Most of the sound recording was done with a stereo shotgun microphone, and most sounds in the world are stereo despite emitting from single points in space (the two channels emitting at a specified angle). One could argue that’s excessive, but to me, it often sounds richer and can give it a light "3D-plus" quality.
For instance, cloth was mostly waved past the mic, so there is usually sideways panning going on within each cloth sound itself, not just the panning due to character/camera movement. Some sounds needed to be mono and precise due to size or distance etc., but most are stereo.
The gates and grates opening and closing are made from the same bits I used in God of War III, trying to be gentler and more elegant. And a lot of the stone machinery sounds of the tower in the third level are from a trip I made to a rocky Malibu shore to record for that game's Poseidon fight. The surf wasn't good that day, but I got a lot of great material rolling boulders down the slope. That tower emits a sort of rhythmic non-electrical "power plant" ambience that I tried to make sound alive and breathing in a sinister, mysterious way.
Gates and Desert Tower Ambience by Journey Sound Design
The cinematic storyboard sequences went through visual revisions and kept getting moved around, and since the sound effects of each was streamed as a single file, the sound kept getting thrown out of sync. So I saved their final pass until the end, and did them in a straight shot over the winter holiday break when I knew they were done changing.
After the final mix in Sony San Diego's swanky mix room during the first week of January, Austin noticed a bass tone I had used in the transitions was blowing out small TV speakers. So I redid all of the transition sequences the Sunday before the final submission in one steady charge. They mostly all use the same elements but each is actually something spontaneously unique.
Story Sequence and Transition Examples by Journey Sound Design
Most of the sounds were hooked into the game by TGC president / creative director Jenova Chen himself. To show him where the different surround ambiences and reverbs should go, I took overhead screengrabs of each level in Maya, highlighted each specific area, and labeled them with which ambience to play and which reverb to use. A picture is worth a thousand words! Sorry I don't have any.
Jenova likes to say that the mountain peak was filled with ocean sounds early in development, but that's not completely accurate. I processed crashing ocean waves to make the faint snow ripple sounds, and they're still in the game!
Read more about:
FeaturesYou May Also Like
.png?width=300&auto=webp&quality=80&disable=upscale)
