GDC 2012: Young, Selfon, Tallarico On Goals For Game Audio

Author: by Staff

To help inspire submissions for the GDC 2012 call for papers, the event's advisory board members for the Audio track spoke out on the challenges facing modern audio professionals, and outlined what they hope to see at the upcoming March 2012 San Francisco-based event. Seasoned industry professionals such as Media Molecule's Kenneth Young, HUGEsound.com's Chance Thomas, Microsoft's Scott Selfon, Brian Schmidt Studios' titular Brian Schmidt, and Video Games Live/Game Audio Network Guild founder Tommy Tallarico all discussed the most significant accomplishments, challenges, and trends facing audio production as part of their drive to encourage submission ideas for the GDC 2012 Main Conference. As GDC advisory board members, these industry veterans oversee the show's Audio track and ensure that each of its sessions remain relevant and hold up the high bar of quality that GDC attendees have come to expect. The call for papers for GDC 2012 will close on September 6, with a list of audio-specific topics available on the official website. In the following interview, the advisory board members discuss key issues they'd like to see addressed at next March's show. What do audio professionals have to keep in mind when working on titles for the web or mobile devices? How do these platforms influence the use of audio? Kenneth Young: Download size and available storage space certainly have a big impact on the approach -- for example, you don't have the liberty of throwing an abundance of streaming music, ambiences or voice assets at the project. On the positive side, such limitations force the developer to really consider whether wallpapering their game with music or thousands of lines of clunky, information-heavy and exposition-heavy dialogue is a good idea. Here's my obligatory Angry Birds example -- it only uses music when it is needed, in the menu and as a payoff upon level completion. Chance Thomas: You need to remember the range of device capabilities. The mobile marketplace has a wide array of devices and capabilities, and today's audio pro needs to understand this range. For instance, a game developed for the hypothetical iPhone 5 will be sold at the same app store that old G3 users are still buying from. Therefore, today's games still have to sound good on the minimum spec platform. For example, a cool interactive music design that works great on the iPad 2 using let's say, three stereo streams, is useless on earlier phones with only a single music stream available. It's important that audio designs still do the game justice on the minimum spec platform. Brian Schmidt: The style of game you'll do for a mobile is most often very different from a console or PC title, as Kenny alludes to. You don't need the big choir-orchestra so much as tasty simple bits to complement the game. In addition to download size, for web-based games, sometimes the developer is extremely concerned with "time to play" -- that is, from the time the user clicks on the game icon, they want to start playing the game within seconds. A game that requires lengthy downloads won't hold the user's attention, and they'll click and go do something else rather than wait even 60 seconds for a download or install. A phone or web game will have a far smaller team and you will probably be the only audio person, so if you're a composer, get used to doing sound effects -- and not hacky, crappy ones! SFX is probably more important than music in many of these games. The SFX for these gmaes are much more likely to be abstract -- or non-diegetic -- than on consoles, so get good at knowing how to inform the player with sound without annoying them. Scott Selfon: You should also aim for agility and "purity" of vision -- game production schedules and budgets mean there will likely be a sole audio creating force. So it's a double-edged sword; often the composer or sound designer will really get to explore and own their audio direction from top to bottom, but they'll also face the challenges we've seen on previous generation platforms -- small budgets, limited time to develop and iterate, working without a functional game build, and a dependence on an often part-time audio programmer to drop in the sounds. The need for mature tools becomes that much more important to get beyond the "sound trigger equals static wave file" implementation. As far as the audio playback environment, there are plenty of different aesthetic challenges -- how critically can the player listen to the audio when they're playing while on a train, or standing in line? Audio often times becomes more of a reinforcement and reward mechanism rather than being used for pure storytelling. On the plus side, mobile devices allow you to practically explore headphones and the high fidelity immersive sound they have the potential to offer; a few mobile titles have even required headphones and been primarily audio-driven implementations, which is harder to pull off when there's a giant television screen or monitor in front of the player. Tommy Tallarico: Mobile device audio is a bit like a reminder of what it was like when a lot of us first entered the industry in the late 80s and early 90s. There's not a lot of space, not much audio driver or software support ... and very small budgets! That being said, some of the most creative and unique stuff is coming out of the mobile category due to the fact that there aren't layers of producers or publishers influencing a certain audio style. So in that space, mobile audio developers should feel free to take chances, to go against the norm and feel free to stretch their artistic abilities to their utmost reaches. I would also recommend that because the budgets are so small, there is no problem with doing it for cheap -- or even free! Just make sure that you are getting some kind of bonus, royalty or back-end percentage, no matter how small. What would you say are some of the greatest recent accomplishments in video game audio, focusing on games in the last 12 months that have used audio particularly well? Kenneth Young: One of the talks with the highest attendance from last year's audio track was Martin Stig Andersen's session on the soundscape composition in Limbo. This is significant not only because it's an indie game (the audio track doesn't get many submissions from that scene, good or bad, so please shake a leg indie devs!), but also because there were a lot of non-audio people in attendance, which I'd love to encourage. The real strength of the audio in Limbo is that it consistently reflects and supports every other aspect of the game and contributes towards a player experience which adds up to more than the sum of its parts. That's something I aspire to in my own work, and it's a real joy to see such a brilliantly pure example of it. I don't think the game is for everyone, but you can't really criticize the audio direction because it's just so beautifully integrated in to the whole. Chance Thomas: I'm not one for uber-violent video games, but I couldn't resist tracking down and listening to the music score from last year's Dante's Inferno. The harmonic richness, tonal palettes, rhythmic interest and sheer ingenuity of that score set new standards for dramatic music in games. It's clearly among our greatest accomplishments in recent memory. Scott Selfon: As Kenny mentioned, I continue to be impressed with the boldness and uniqueness of indie titles and their ideas of how to make sound a key gameplay component. We see great and ever-increasing sophistication of the epic scores for AAA titles, but I love to see games push into new musical and sonic territories. In addition to Limbo, Papa Sangre was a great mobile title that told its story nearly exclusively through sound. And the Independent Games Festival saw numerous unique and innovative uses of audio; that continues to be one of my favorite areas to visit at the Game Developer Conference. Tommy Tallarico: The interactive audio experience in Red Dead Redemption has got to be one of my favorites. Rockstar seems to always take the time, make the resources available and spend the money to ensure the highest possible audio outcome. L.A. Noire is another great example. But the interactive music, sound design, voice-over acting and overall audio integration and mix was absolutely phenomenal in Red Dead. How important is it to be a "game-only" musician or sound architect in today's industry, or is the market increasingly cross-platform across TV, film, etcetera? Kenneth Young: I don't think it's ever been important to be "game-only," but I do think it's very important to understand the challenges that games and interactivity pose. To that end, because the general complexity and sophistication of games is increasing, you'd think the market would favor those people with experience. And yet there is a trend for film composers with zero games experience to score AAA games... Chance Thomas: I've long observed that the most immersive and gameplay-relevant scores are typically done by composers with veteran game development experience. Which isn't to say newcomers can't have good instincts. Some terrific fresh ideas have come from people new to the business. Also, experienced audio leads can take stems from film and television scores and create decent game scores with them. But by and large, the more experienced game composers are the ones who think most adaptively when they write, and tend to deliver the most riveting interactive music as a result, which shouldn't be surprising. Veterans have been marinating on these challenges for a very long time. And they've seen lots of ideas come and go over the years ... Direct Music anyone? I think there is tremendous value in a dev team partnering with an experienced game composer if they want a truly relevant score. Brian Schmidt: I'll echo Chance and Kenny. I think if you want to do a really good job at game music and don't know games, then you'd better team up with someone who does. Would you score a movie with a composer who didn't watch movies, didn't know other composers' film scores? No, you don't have to know FMOD or what streaming is in order to write great, emotional music, but how can you create art for a genre that you don't know at all? Scott Selfon: Game composers and sound designers exercise many skills that come in handy for (and can make them more successful with) linear media, but I agree with everyone else -- understanding and awareness of games, where they've come from, how they're experienced by players, and the conventions of audio for interactive media are all prerequisites to fully exploring the medium. It's certainly welcomed and encouraged to break some of the rules -- and awareness of those rules allows for the most effective and unique boundary pushing. Tommy Tallarico: Being passionate about video games and understanding the technology of interactive scoring and sound design is a must for creating successful audio for a video game, but as Brian said, any film or television composer could hire someone to help them out. It's not just about writing music or doing sound effects anymore. Integration, interactivity and knowing what the platform or technology is capable of makes a huge difference. Having someone on your team that knows the ins and outs and understands the video game world is a major plus. I'd love to see a guy like John Williams do an original score for a video game, but he would need a seasoned audio pro from the game industry to put it all together, get it interactive, and make it the best it could be. What do you think are some unexplored avenues for games that rely heavily on music or sound? Brian Schmidt: There are definitely unexplored -- or lightly explored -- areas of game sound. For example, tightly coupling audio with physics, direct synthesis -- the physical modeling of sounds. We keep hearing that the power of these new consoles may lead to a re-birth of the MIDI synthesized score, perhaps with instrument-based controllers as input devices to obtain more performance nuance than is possible with keyboard input. And to this date, by far -- by literally an order of magnitude -- the most attended talk at GDC by a game composer has been Koji Kondo, who does MIDI generated music for his games -- he believes it essential to the aesthetic. Using sound to propagate game information in a more realistic way is something that needs to be explored more. A recent Gamasutra article on AI lamented the fact that the speed of information propagation to the AI characters for games is infinite. Yet in the real world, a sniper bullet wouldn't be heard until a significant time after impact -- often for several seconds, so why do the AIs instantly know that your rifle was fired from a thousand yards away and start firing right away, instead of the 2.5 seconds it would take for sound to travel to the AIs ears to get a bearing on you? Hmm…maybe I should submit a programming track talk entitled "Using the Physics and Psychoacoustics of Sound to Constrain World Information Propagation Velocity in AI Systems"... Scott Selfon: Brian, let's do it! If I could rant about audio interaction with weather systems a few years ago, we can certainly challenge some of the psychoacoustic rules that are consistently broken in games. As far as music, getting beyond highly rhythmic and tempo-locked loops is an ongoing challenge; tempo is often one of the most dynamic tools available for changing the emotional context of a scene -- Asteroids, anyone? Yet we've in many ways lost that ability in the era of the pre-recorded score. For sound design, Brian mentioned one of the top challenges: infinite material-on-material sound interactions, and making everything sound like it's real ... or, depending on game aesthetic, hyper-real. Overall for game sound implementations, I still think we too often get bogged down in the literal audio simulation, creating sound and music that's frequently "on the nose" rather than using sound to help tell the story, inform the player, and influence their emotional status -- even potentially playing against the visuals. Lastly, as to dialogue, I'm still hoping for more intelligent and context aware characters. Dialogue is still too often seen as just a list of one-off triggers, and if actors are delivering lines out of context, it's extremely challenging to get convincing performances. Characters all too often speak in a vacuum, repeating themselves verbatim, speaking over each other, using awkward cadences, and not really making me believe they're real. This is our core method of interpersonal communication in the real world, yet it feels like it's far behind both sound design and music in interactive media. What subjects are you particularly excited to see covered in this year's Audio track? Chance Thomas: I want to hear successful veterans talk about how they have managed to build career longevity on such a potentially slippery career path. Game development takes place in a dizzyingly competitive and notoriously fickle marketplace, where new technological breakthroughs create market sector obsolescence regularly. Obviously, it can be done. But the question is, how does a young aspirant build his or her own strategy for long-term success in this business? The idea of having a successful career as an audio contractor in games is incredibly alluring. But the reality is, it's a tough place to practice your trade. So how do we best equip the rising generation of sound designers, composers, mixers, voice directors and audio programmers for success -- educationally, strategically, emotionally, ethically? That's what I would be particularly excited to see covered in this year's Audio track. Brian Schmidt: I like Chance's idea there. It'd make a great panel session -- similar to the "In-House or Freelance" panel we had a couple years back. "Making a Career of Game Audio" would also be a great bootcamp talk that I'd be happy to give. I'd like to see how games are starting to use non-wave based audio, particularly in huge open world games where the sheer amount of wave files needed to cover every object and object collision is immense and impractical ... the latest Crackdown apparently did a bit of this. I think it might be time for a "Business of Game Audio" session again. Many composers don't seem to know that playing WoW online doesn't mean the music is streaming, and therefore not due any kind of performance royalty. And let's not downplay the news that Lada Gaga had a song premier in FarmVille! Scott Selfon: What do the next generation of games really need from sound? And I intentionally use the phrase "next generation" at a time when we're seeing many new, different, and enormously popular platforms coming to the forefront at the same time: a bustling indie scene, high-definition and rapidly evolving mobile, the democratization of game development, games as (and in) social media, cloud-based computing, natural user input mechanisms, and more. I'd love to see people with early experiences on each and every one of these topics, as well as how people change their work, business, and aesthetic styles with each of these. I want to explore what can we deliver that isn't being done today that will delight players, enhance or redefine the genre, grow the business, and of course allow the industry to support the livelihood of talented audio creators. Brian Schmidt: And as a general aside, I would really like to see if we can figure out a way to "un-silo" audio and other disciplines from the rest of the attendee list. Many "audio issues" are really game production issues more than audio issues. In general, I'd like to see some cross-discipline focus, where there are specific tracks or sessions called out specifically as "Enhanced Discipline" sessions. Scott Selfon: Hear, hear! While each discipline does have its own unique challenges, many are shared, and many more could benefit from a new perspective or cross-discipline education. I've seen audio tackle parallel challenges to other disciplines or struggle with a topic that could potentially be addressed cooperatively, such as dialogue, physics-based sound implementations, and so on. Deep dives into the nuts and bolts of audio are certainly hugely valuable, but taking advantage of the fabulous, talented, and diverse community brought together at the Game Developer Conference to tackle issues and raise the quality bar across the board is a real opportunity. Additional Info GDC 2012's call for papers for the Main Conference is now open until September 6, with Audio-specific topics listed on the official website. Potential speakers are encouraged to send in their session proposals via GDC's online submission system -- the event itself will take place March 5-9 at the Moscone Center in San Francisco. For more information on GDC 2012 as the event takes shape, please visit the official GDC website, or subscribe to updates from the new GDC news page via RSS, Twitter, or Facebook. GDC Europe is owned and operated by UBM TechWeb, as is this website.