Tuning Canabalt Part 2: Audio

[This is part 2 in a 3-part series looking at how the one-button runner Canabalt was created for the browser and ultimately sold on the iPhone and iPad.  Part 3, focusing on the iPhone port and biz aspects, will be out in a few weeks I hope.  This post was syndicated from our company blog at http://blog.semisecretsoftware.com/tuning-canabalt-part-2-audio, where the video and audio embeds are automatic, haha.  Onward!]

When I attended TIGJam Winnipeg, an indie game jam in Canada in Spring 2010, I talked to the lovely James & Lisanne (Indie Game: The Movie) a little bit about Canabalt, and ended up talking a bit about the sound of the game and how it was built:

http://vimeo.com/14125649

I wanted to expand on those thoughts here, by first exploring the process for creating the sound effects in the game, and then by posting a short interview I did with our musician Danny Baranowsky (Super Meat Boy, Gravity Hook HD) to cover the music half of things.  One disclaimer though: I am not nor have I ever claimed to be a competent sound engineer!  There are professionals out there who do this for a living, and do it well, and write about it a lot.  This is just the short story of how I stumbled through the process at the last minute.

Sound in Games

Growing up, I had a really weird selection of NES games.  We bought all our games with our meager allowance (it usually took 3-6 months to save up for a new game), so we mostly rented games from the grocery store, stocked from a shelf with some VHS tapes and X-Men comics.  Sometimes, if we were lucky, we'd get to rent a game from the weird red sheet-metal barn up the street from the grocery store.  I didn't get gaming magazines or anything, so my exposure to games was very much a word-of-mouth kind of experience, combined with said grocery store's selection.  Nevertheless, I slowly amassed a semi-eclectic collection of NES and eventually Genesis games that I really adored.

Of course, when you're buying consoles with your allowance, there's a lot of encouragement to garage sell your existing collection to help finance the new one.  So when the PS1 was released in the US, I'd already been saving for months, and had auctioned off my Genesis and the few odd games I'd had for it ages ago.

Fast forward a decade to the spread of broadband in the late 90s, and with it, emulators and ROMs.  Suddenly, games I'd casually disposed of were accessible again, and I was seeing them completely fresh for the first time in years.  Naturally, I first gravitated toward the games that I'd adored as a kid; I couldn't wait to see what they were really like.  It was time to remove the rose-tinted glasses, and dive into these things with my eyes (and ears) wide open.

It was a shock to no one, least of all me, but a lot of the games I loved when I was growing up were pretty much crap.  Some of them were really great, but a lot of them were definitely garbage.  Except for one thing - the music.  Nearly every game that was on my nostalgic list of beloved titles had spectacular tunes.  A few selections:

http://www.youtube.com/watch?v=hiXUt9RdIPI&feature=related

http://www.youtube.com/watch?v=T4WPWa6WwlU&feature=related

http://www.youtube.com/watch?v=7SA_DarPYl8&feature=related

http://www.youtube.com/watch?v=4KcY745Qbd0

Some of these games were actually pretty good, some of them... not so much.  But I still love how they sound, nostalgia or not.  I rediscovered this stuff about a decade ago, and it had a really big impression on me.  I realized that sound in games isn't an afterthought or a luxury, but a huge part of the experience, and not something I intended to take lightly.

Canabalt's Sound Effects

That said, sound was something I more or less forgot to address in a serious way until about the last day of Canabalt's development.  On previous games the amount of work required to produce the sound effects was microscopic, as I was relying on SFXR to generate charming and usable "8-bit" effects for games like FATHOM and Gravity Hook.  SFXR is a phenomenal synthesizer for simple square, triangle and sine-wave based sounds, with great presets, and really useful mutator and random functions.  If this is the sound you need, SFXR is definitely the fastest, easiest way to do it.

But that's not the sound I wanted for Canabalt.  While I was building it I could actually hear how the game sounded, and it didn't sound like its inspiration Super Mario.  It sounded a lot more like a movie or a modern game, believable sounds that ground the action and the events on screen to some kind of reality we automatically empathize with.

So, on the second to last day of development, I wrote up a short list of effects I thought I'd need - here is a rough list I pasted to Danny while we were discussing sound that afternoon:

With about 30 hours of development time remaining (assuming no sleep, but I'm not that hardcore), this was apparently encouraging and not intimidating at the time.  We actually went back and forth a bit on how these things should sound - I was certain I wanted believable, gritty sounds, but Danny was not exactly buying into my vision.  At the beginning of the sound design process, Danny actually engineered a "flyby" effect, played when the jet rushes by in the background.  Here's his version:

http://adamatomic.com/pics/blog/canabalt/audio/flyby_danny.mp3

In the meantime, I was trying to come up with something that fit what I was hearing in my head:

http://adamatomic.com/pics/blog/canabalt/audio/flyby_adam.mp3

Here's our conversation after I posted a build with just a few of my effects in it, including that one:

I stayed the course, for better or for worse.  I had never done non-8-bit-inspired sounds for a game before, and was kind of at a loss about how to continue.  I ended up using two basic methods to rough out this effects list, and then a simple last step to get the quality where I wanted it for the game.  The flyby was created using this first method.

Search and Destroy

This approach to roughing out the effects involves lengthy, painstaking, and supremely dull google searches for free stock sound effects that I can distort, layer, or otherwise manipulate into the thing that I want - the interview with Indie Game: The Movie covers that process a little bit.  I use a free, open-source, cross-platform sound editor called Audacity to do all my (admittedly amateur) engineering work.  The main functions I found useful were Bass Boost, Change Pitch/Speed/Tempo, and Reverse.  I also used the mouse to highlight and delete chunks of the sounds that I wasn't interested in.  I used this basic approach to rough out the sounds for the spaceships flying by, the buildings crumpling, and the windows breaking - pretty much anything that was too difficult or dangerous to do the other way.

Crappy Home Foley

This is the other way, and it's way more fun than the first method.  You take your laptop's built-in microphone, and you gather up all the crap you think you might need, then you bang things together or throw things at other things over and over.  It's cathartic and useful.  I used Audacity to record the effects, and I usually performed the same effect 8 or 10 times in a row on the same take, then put on headphones and listened to each attempt, and tried to select the best ones, or else started over with different materials.

To get a good hollow, ringing sound for the player running along the crane in Canabalt I used a steel pot and my own running shoe, which doesn't require a lot of imagination really.  I did originally try shoe against cookie sheet, but it was too "bangy", and just didn't sound right.  The doves and the sound of the player tumbling and rolling involved blankets and couches, while the obstacles were actual cardboard boxes being tossed around.  A good mix of practical and clever ideas seems to be able to really do the job when you're approaching your sounds this way.

Polish It Up

Finally, regardless of how the rough effects were sourced, I used a similar process to get them ready for the game.  First, I used Audacity's excellent Noise Removal feature.  This is another good reason to record 8 or 10 effects in a row - you can sample the stretches of sound wave in between effects to get a better noise profile for the Noise Removal effect.  It's very easy to use - you just highlight a bit of "silence" and then highlight the section you want to "remove" that noise from.  This will in some ways damage the sound, but the gains outweighed the losses for me.  Depending on the sound, I would usually do a little bit of Bass Boost on it, just to give it more presence.  Not everything needs that, but it was coming up a lot.

Finally, as I mentioned in the video, I fiddled with the panning, or the degree of left- and right-ness, of the effects.  Canabalt's presentation was unique in some ways - the player was always oriented to the far left of the screen, while obstacles always appeared from the right and proceeded toward the left.  This let me make a bunch of very simple adjustments to the apparent location of the sounds without ever writing a line of custom sound code.

Almost right away I learned a tough lesson though (including having to re-record some sounds I forgot to back up).  Even though the player is on the far left side of the screen, I absolutely did NOT want the footsteps to be panned all the way to the left.  My first pass at this made all the effects sound like they were coming from 3 feet to the left of the screen, it was really terrible.  If panning is a scale from -10 to 10, with zero being dead center, I ended up panning most of the player sounds to about -3 or so I think.

I almsot forgot to add that there's still another reason to record a bunch of attempts at any given effect.  For any noise that's used a lot in your game, it can make a huge difference if you select from a few different variations of those sounds at runtime.  For example, Canabalt uses 4 different footstep sounds for the roofsteps, and another 4 sounds for the crane.  Each time a step is about to be played in the game, I randomly select one of those 4 sounds to play back.  My favorite example of a console game doing this is probably Castlevania: Symphony of the Night, in which Alucard has 4 or 5 different grunts (including just silence) that he might emit whilst swinging his sword.  Since you swing your sword 10,000 or so times in any given runthrough of the game, it really benefits from a lack of predictable repetition there.

The Tunes

I was starting to get pretty excited about the game as I worked through my list of a dozen or so sound effects, panning them just so, cleaning up the noise, and dropping them in-game.  As soon as I had them all in, I sent Danny a new build:

Needless to say, I was basically floored by Danny's first pass at the music.  There's not really a lot I can add to that initial response, so I'll just end this post with my short interview with Danny about the process of creating the memorable track.

Danny's Take

Me: How long did it take to write the original track?

Danny: The original track for Canabalt was written within the space of a day. The power went out in my studio, so I wrote the first half of it on the couch in the living room. Once my power was restored, I stayed up until about 5 am finishing the track. That became "RUN!" (the track from the Flash version).

Me: And what hardware and software did you use?

Danny: I use Propellerheads' Reason for 90% of my music. I use Steinberg's Cubase for tracking audio and advanced sampling (mostly orchestral). The music in the first Canabalt track was all Reason except for the guitar track, that was tracked into Cubase. I use a korg nanoKEY to audition samples, but 99% of everything I write is ultimately clicked in.

Me: What was the creative process like?  Did you have any inspirations in mind?

Danny: Interestingly enough, the number one consideration I had for the music for Canabalt was that players could hear the fluttering dove sound effect you made. I started out in indie films, so I really tend to go for a cinematic presentation whenever appropriate. As I was playing the build you had set up for me, I felt the scene of the running man dashing through a flock of departing doves was breathtaking. It was insanely cinematic and emotive, especially for a pixelated scene. My number one priority was to try and enhance that aesthetic. So I started with an energetic, simple electronic riff. I definitely had the Matrix in mind (this is funny, I absolutely despise that movie - Adam), the color scheme and subject matter seemed to suggest that kind of approach. But again, the ideal situation in my mind was this scene of a super energetic run across rooftops, and then the music subsides perfectly to attenuate a flock of doves swarming above you. Having played the game a lot, I can attest that this does indeed happen :). I guess the best way to explain RUN! is that I wanted a track that would enhance the immediacy of running for your life from evil space robots, but also the sad beauty of a burning city. It doesn't get more pretentious than that!

Me: Did it help to see early versions of the game, or would you have preferred to see it fresh near the end of development?

Danny: I'm not sure about the effect of seeing early versions of the game. One of the early prototypes was a box jumping on other boxes. For some inexplicable reason, it made me think of Labyrinth Zone from Sonic the Hedgehog.  You probably thought I was fucking crazy. Regardless, I place a lot of weight on understanding the "feel" of a game to write an appropriate score. Knowing the general game elements allowed me to roll ideas around in my head for days before seeing the art. I think it's far more important to match the gameplay than the aesthetic. The aesthetic of a song can be changed somewhat easily, but the underlying feel is  the core of the experience, and has to be dead-on.

Me: What would you have written if we went with the 8-bit sound we considered?

Danny: I think Canabalt could have worked with an 8-bit score. I think it could have worked well. But I'm not sure it would have been better than what's in there now. It's a hard to answer, alternate-universe kind of question. It certainly would have felt differently. Chiptunes tend to elicit a certain kind of nostalgic response. I love chiptunes, but I think the cinematic route was the right way to go. Also, I think pixelated games are expected to have chiptune soundtracks, and it's fun to play with peoples' expectations sometimes.

Me: How did the creative process for the other two music tracks (on iPhone/iPad only) compare to the original?

Danny: When Canabalt was first released, a number of reviewers said it had a "chiptune" soundtrack. I can tell you with 100% certainty that RUN! is not a chiptune. The term has been thrown about and so broadly applied that it's nearly meaningless anymore. My goal with the second track was kind of "try and call this a chiptune". So I went 100% orchestral. Nobody called it a chiptune. The third track was essentially an attempt at a middle ground. Something cinematic, but also a throwback to retro-times.