Opinion: The role of technical sound designers

[In this reprinted #altdevblogaday interview, Volition's Ariel Gross talks with veteran audio contractor Damian Kastbauer about the important role technical sound designers play in game studios.] I caught up with Damian Kastbauer, technical sound designer, in the sticky jungles of the Congo last week. He was questing for the fabled paisley hippopotamus. Legend says that when the paisley hippopotamus is kissed upon its patterned lips, the kisser is granted a treasure of immense value. I found him hiding behind a giant leaf, wearing a giant leaf. He was pressing binoculars against his eyes, gazing towards a nearby algae-covered pool. I crouched next to him and started asking him questions. Ariel Gross: So, Damian, what is it that you do? Aside from questing after fabled beasts of yore? Damian Kastbauer: I'm a technical designer for games, which is a somewhat nebulous term. It's defined pretty well by Rob Bridgett in this article. Essentially, I try to serve as a bridge between sound content and programming/engine-side integration in order to create systems for sound playback, or just plain getting sounds in the game working and sounding right. AG: Nice. My opinion is that sound design in games is only as good as its implementation. What do you think about that? DK: It's often said that the best sound can end up sounding bad if it's not implemented properly, so, a lot of the time I'm trying to let the sound do its thang… which is not always easy. Since I don't create content, and I'm not a programmer by definition, I usually spend a lot of time either building tools, streamlining pipelines, and strategizing audio features with the help of the programming team or working with audio middleware tools to similarly create the smoothest integration possible. Damian finally released the binoculars, which then fell and dangled around his neck, but kept his hands in place, and continued to look off towards the pool with his hand-noculars. It was an interesting technique that I had never seen used in the field before. He made a refocusing gesture and continued to peer out through his hand holes. AG: So… do you find yourself being a middle man between Team Audio and Team Programming? That could be challenging, what with all the unrequited love and/or red-hot rage between those two disciplines. DK: Definitely, I'm helping to bridge the gap between the two worlds. Whenever possible, I like to enable content designers to just "design in the box" all day from the DAW without having to worry about the sometimes-labyrinthine pipeline to get sound into the game. In this way, they can focus on what they're really into and I can go to town with making sure it all works within the context of the game. It's a balance that requires human interaction, but I'm pretty keep on that as well. AG: Wow, you just said labyrinthine. I'm golf clapping right now. In my head. And in my heart. I don't want to scare off the paisley hippo. You're welcome. So, have you found any common issues between Team Audio and Team Programming among multiple developers, and do you take measures to help both sides see eye to eye? DK: I think one of the missing or underdeveloped pieces of some studios is the direct communication between Team Audio and the rest of the teams. It's common for audio to touch almost every corner of the pipeline and development process. With Team Audio often sequestered within a sound-proof cave deep within the bowels of a studio, it can be hard to get the kind of happy accident or magic moment that can come from being out in the pit or lined up in a hallway. AG: This is painfully true. DK: One of the things that has always seemed right about my involvement with different game studios on-site is the necessity of being "on the floor" with the other disciplines… which has always been a positive in bridging that gap. When I work remotely, I put a tremendous amount of effort into achieving a high level of communication between the different teams on a project. Whether it's e-mail, IM, Skype, or occasion on-site visits during development, the communication between people is among the most important aspects of working on a project. If I am outside the fence, having an on-site advocate for audio is a definite plus, and in most cases, it is key to making my role as a facilitator for sound actually happen. Some foliage near the murky pool began to shake. Damian dropped his hand-noculars and placed one hand over his mouth to ensure complete silence. He placed his other hand over my mouth, too. I resisted the urge to lick his hand. A rabbit bounced from the foliage. We both sighed deeply. AG: Miv viff fuhm coffm prbflrm… Damian removed his hand from my mouth. AG: Is this a common problem at studios you've worked with? That Team Audio doesn't pop their heads up through the manhole enough to be part of the greater discussion? DK: Whew, well, I think that as a matter of course it is hard to have visibility across the team in the same way as being out in an open floor plan, when part of your daily work requires you to be locked away in a padded room. That is, people can't SEE you and demand your attention by rolling up to your workstation. I mean, there's a door… you have to turn the knob… like, maybe even knock… and then get blasted in the face by high decibel explosions in order to interface. We know how averse most people are to moving in the first place, let alone opening a door. I chuckled. DK: A little dry, mid-west sarcasm for you. I chuckled more, leading to a whole-body belly laugh, leading to a maniacal, snorting cackle, ending finally with a very feminine giggle. Then we sat motionless for what felt like an eternity. AG: Sorry. What you say is completely true. Before tracking you down in the Congo, I hadn't left my seat in four years. Not even to use the restroom. I held it. What you saw at the GDC was my hologram. A hologram with mass. I digress. What is the number one reason why a studio hires a technical sound designer? DK: The number one reason I am hired is to help ship games. Having shipped my fair share, I would like to say that I know all the tricks in the book, but this is absolutely not true. What I hope that I bring to a project is an ability to work together with people to get things done, through thick and thin, regardless of the task at hand. Sometimes I don't even know what I'm going to be doing until I get involved and assess the situation… this can be true of the studios as well. They're not sure what exactly needs to be done, but they need someone to help pull it off in the eleventh hour. AG: You're basically a hero that is brought in to save the day. DK: If I can come in and take the burden off of content designers, help design complex playback systems for physics or procedural animations, help wrangle memory budgets or streaming look-ahead times, then that is one less thing for someone, who already has too much on their plate, to worry about. For remote work, it can be at any time during the project, I work from the home studio connected to a VPN using source control while making local builds from day one. When I do work on-site for an extended period of time, it's usually toward the end of a project for a few months. At this moment, the waters of the pool began to ripple as a large, paisley-patterned mound broke the surface of the water. Then, two paisley ears emerged, followed by blinking eyes. Damian fidgeted with excitement, but stayed put. AG: You mentioned memory and streaming. This is something that some people don't realize. Someone on Team Audio is usually tasked with wrangling memory budgets and streaming bandwidth. It's really techy work. This isn't always expertise that is considered when hiring an audio designer. I've found that studios tend to focus more on whether or not an applicant can make rad sounds, with less emphasis on the technical side of things. There's usually some requirement that says, "familiarity with middleware tools and implementation," but I get the sense that the weighting at most companies is 80 percent creative aptitude and 20 percent technical prowess. Does that seem accurate to you? Or am I totally wrong? And ugly looking? DK: I think there are a majority of people who come to studios with a sound design background. Until recently, it has been almost non-existent to have someone come out of school with an education in the technical side of game audio. It used to be that "audio implementer" was the job that someone new to the industry would take on their way to becoming a sound designer… or even a musician or composer for games… that was really the only way to learn the ropes. When I got into games, it was clear to me that the technical side was the only thing that I wanted to do. Thankfully, in the past few years, it has become a niche that has benefited from a group of people whose interests are purely in the technical, which has helped to establish legitimacy for the specialization. AG: We used an in-game editor to place ambient audio emitters on Saints Row 2. Sometimes the frame rate would tank while I was moving an ambient emitter around, and I'd fling the emitter object 100 miles away, sometimes into a different game entirely. I think a few of those emitters landed in Red Faction: Guerrilla. DK: Hahaha… user interface physics lerping… TOOLS! AG: Do you ever find yourself caught up in the mystery of missing sounds? Like, the audio team is sure that a sound should be playing, but it's not. Is that something you run into a lot? DK: Totally! Someone did a model swap but forgot to bring the audio hooks with it, or moved the entire level geometry across the level away from the audio emitters that were placed on a different layer in the editor… happens ALL the time, and the resources to figure it out are sometimes underestimated. With all the different ways that people can approach game audio these days: mods, Unity, UDK, FMOD, Wwise, SDK documents, Max/MSP or pd, it's becoming common to have fresh faces joining the team who already have a good handle on the tasks at hand and are ready for the challenge. The fabled paisley beast began to slowly emerge from the algae-covered pool. Watching the water cascade off of its psychedelic hide was glorious. Swirls of color and waves of sweet smells washed over me. Granted, I had also been munching on some random red berries that I'd found on the ground. AG: I know plenty of sound designers that don't want to do the implementation. They look at it as a boring chore. A chore-bore. A borch. Anyway, I could see some Team Audios out there wanting to hire you on full time. Do you ever get this? Have you considered an in-house gig? DK: I've worked remotely on a couple of projects for a longer duration (1-2 years) serving as a conduit for getting content in the game. I've worked a ton with Bay Area Sound as an outsource audio solution involving their audio and music content creation in conjunction with my management of the technical side in what has been a perfect solution for smaller developers and teams that need supplemental assistance. I worked for five years in this capacity on the games for Telltale. I integrated content provided by Bay Area Sound and Harmony Machine into The Saboteur from Pandemic Studios for about a year to help augment their in-house audio team, which included building vehicle systems in Wwise to work with the simulation team who was responsible for the cars driving around in the open world. Most recently I've been working again with Bay Area Sound to integrate content and tool the pipeline for a couple of MMOs using the Hero Engine in conjunction with FMOD. AG: How about documentation? I have seen with mine own eyes the lack of documentation out there. Do you find yourself having to trudge through undocumented systems a lot? Do you end up documenting them yourself as part of your services? DK: Documentation is always a tricky one. A lot of developers who have been iterating on in-house engines and tool sets carry with them a ton of institutional knowledge… usually within their brains. There's usually a pretty intense time of education at the beginning of the project where I do my best to document… for my own sanity as well as anyone who might find themselves in a similar position in the future. I recently worked with a company and provided an evaluation of their audio middleware integration in addition to recommendations for how to move forward with some key changes. In that case, it was documentation and justification for the different recommendations, as well as education in order to get everyone thinking about the possibilities. AG: Do you find that most companies that you work with have an audio programmer on stand-by to compliment you? Compliment both in terms of assisting you in a complimentary fashion, and also to tell you that your shirt looks very nice tucked in. DK: It's definitely becoming more common for developers to recognize the need for audio programming support throughout the project. While I think it's becoming, I think there is still a false expectation that you can just throw on the UI programmer for a couple of months and sew everything up. The need for dedicated audio programmers is definitely growing as we continue to scale in meeting the demands in quality during the current generation. So, I guess, having an audio programmer is necessary to compliment the technical side of sound design, especially a dedicated professional who is invested in sound. Their ability to bring not just programming, but a knowledge of audio to the table can only mean an increase in the quality of the audio and the quality of life for Team Audio. That, and yeah… someone to tell me when my dashiki doesn't match my socks. AG: Last question, and then you can get back to your quest. What do you think will change from a tech audio standpoint with the next generation of consoles? DK: I think the industry at large will further close the gap between pre-rendered and in-engine… across all disciplines, a continuation to move forward into real-time. What this means for audio is building on the sample-based methodology that has firmly taken root in the current generation while using more procedural audio, synthesis, and DSP to modify or accent sample-based content dynamically at runtime. Additionally, increased transparency and better communication in both directions between the game audio and audio engines will develop, which will in turn necessitate a further push towards enabling this functionality within usable tool sets. This accessibility will reach into every corner of the audio production pipeline, simultaneously exposing the technology while making it easier to adapt it to the interactive needs of the game. Hopefully the new consoles will also make espresso. AG: Espresso feature would be cool. I'm hoping that someday I can feed a slice of bologna directly into the console. It would recognize the meat as bologna and procedurally create a game called Bologna Quest, where a young adventurer named Ariel, clad in the finest bologna armor, would climb the mountain of… Damian? Damian, recognizing that I had started my typical bologna-related sign-off, was already planting a kiss upon the lips of the sleeping paisley hippopotamus. Then, a flash of light, and they both vanished, never to be seen again. [This piece was reprinted from #AltDevBlogADay, a shared blog initiative started by @mike_acton devoted to giving game developers of all disciplines a place to motivate each other to write regularly about their personal game development passions.]