Design's Future Potential Engaging Audio

I have been buoyed recently due to some excellent writings by Ariel Gross and Rob Bridgett talking about game development, and also by my own sense of belonging in the world of games. We are at such an exciting and interesting point in history as it relates to games, and there are practices and stylistic methodologies that deserve some discussion.

I mention Rob Bridgett, as he recently published a blog post about the changing role of the sound department in games. He sees sound department’s role as “principle collaborator to not just the overall project, but artistically, technically, socially and politically in the development of company culture”. This statement is one that rings true, particularly now. I think most game devs will recognise this, and game development companies the world over, if they are not already doing it, will soon be recognising the people in audio departments willing to put up their hand to fill these roles.


We can see that games are growing up. I have always loved games, playing them, and more recently helping build them. And for me, right now is the most incredibly exciting time to be involved in both these aspects of games. The recent events in games such as #1reasonwhy, the success of Journey, of Kickstarter funded games, of Australia finally passing R18+ ratings, the success of story in The Walking Dead, Indie as King, violence being seriously questioned as the primary mechanic in games. This is an incredible time.

Finally games emerge, and it begins to be normal for them to explore more interesting terrain. And I get to be a part of this. I get to help shape the next games. Audio team members with skills and perceptions and experience to share are capable of contributing to game development. We all have different sensibilities, styles and skills, but all are valuable. What follows are some ideas from audio that may be interesting to consider.

There have been many examples of exemplary sound design for games recently. Limbo, Journey, Bastion, Dead Space, Sleeping Dogs etc. But although games can do amazing things in different ways than other mediums, we can still learn from other mediums, in much the same way other mediums are informed by each other. There are techniques that translate, ways of thinking about character and point of view and drawing in a viewer that other mediums such as film have experimented with, and that games can just as easily borrow from.

Films have been at it for a while now, and have experimented with their medium in amazing ways, providing us with some rich experiences from which games can borrow. Games already use many film conventions; in their camera views, pacing, every cutscene. Sound is no different. But as I am more familiar with sound, I can see where sound in games has not yet managed to strive for the depth and subtlety and innovations that enhance plot and character and ultimately experiences, that film has already explored. For example the technique of fading to silence, already employed in games, was used in the early years of cinema in ‘Ikiru’, by Akira Kurosawa. The fades to silence are so subtle and beautifully unexpected, that no cliched use since has managed to replicate the sensation quite so well. The use of silence can be powerful. In modern games the ‘Saving Private Ryan’ model is used to the point of major cliche, but there are other more subtle applications for this technique, that with the right team could lead to magical, beautiful gaming moments.

The use of silence, or near silence, to allow for beauty is also seen in ‘Let the Right One In’. There are many scenes where the subtle foley sounds are the only source of sound, and this strengthens the sense of isolation, humanities coldness, and sense of familiarity between the main characters. It is a technique that many games could easily use, if discussed early and the games are designed for this technique. The sensation of quietness and subtlety given by focusing on the small sounds sucks you so completely into the world, and allows some idle thought to go into imagining the world outside the space depicted onscreen.

When talking about the ability of subtle and sophisticated use of sound to pull an audience in, I often think of Tarkovsky. In every film I have watched of his there are extended sequences full of the most beautiful, subtle sounds. There is also usually mechanical noise present, much like tape hiss, or the sound of the projector, which seems to add a documentary quality to the action. But there is always an emotionally overwhelming sense of beauty and humanity in his work. I believe games have just as much potential to explore the human experiences as Tarkovsky’s examples, and failing to reach the height of Tarkovsky would still yield a richly rewarding experience.

Tarkovsky wields score with a deft magical touch. This is fairly rare - it seems that many times in films as in games, score is relied upon too heavily. Let me posit something to you. This is an opinion that ever since I heard it has stuck. I cannot and will not shake it. Imagine you are reading a written story, and as your brain and heart feels it, another layer begins telling you, in parallel, how you ought to be feeling. This is too often what score does, and this immediately says to an audience “this is how smart I think you are. I cannot trust you to feel appropriately, so here, this is how you must feel”. This is by far and away the most common use of score in games, and is a lowest common denominator approach, and at worst immediately insults your audience. The only use of score more despicable than this is as wallpaper. If you can’t sustain interest with the design of your game, using some muzak to distract is a lazy and cowardly device that saps the potential of any future emotional scoring you may attempt. It is a pervasive convention, and I forgive anyone guilty of it. But be aware of the effect!

Of course the above uses of score are also the most common uses of score in film. But with film, it is a more experienced medium, and has explored some brilliant methods of treating score. As a modern example, ‘There Will be Blood’ has a quite unnerving score. This is somewhat of an understatement. It is used sparingly however. It might only present itself in 15 mins of the film. For the rest of the film the amazing performances and great sound is all that’s needed to hold rapt attention. Can games rely on set pieces, performances, pacing and design to hold attention? Of course they can. It just takes brave developers working in tandem with all their departments.

‘Cast Away’ is another film that manages to be stronger for its restraint in use of score. For the entire first half there is none whatsoever, and when it does finally make an appearance, the previous lack of score strengthens the emotional resonance enormously. Philip Brophy describes the purpose of this in his article ‘Human Cargo & Sonic Flotsam’. “The film has deliberately refrained from using music so as to mark this absence of humanist commentary on Noland's plight” This idea of manipulation by the score being deliberately withheld as a device has some interesting applications for games.

This next film is probably the most famous example of a similar technique. The opening of the film spends time dwelling on the characters and the sounds that surrounds them before introducing score to brilliant effect. It is Sergio Leone’s ‘Once Upon a Time in the West’. This technique has probably already been borrowed by games, but this early example is amazing, and if you haven’t yet seen it, is a must watch. The use of score as character in this film is also extraordinary.

All Leone’s films deal with sound in a considered way, using it to add to the story. The above opening sequence is a great example of how sound and score can be used together. Good sound design can be utilised to not only describe what is happening, but it can also be used as a character. It can have the qualities of the character - rough, gritty, rugged, or soft, wispy, ethereal. This is fairly elementary, but what if the same sounds are perceived by these characters differently depending on their emotional state? This is how point of view operates, and needs to be set up early by all departments. It is a technique eloquently described by Randy Thom:

“Most of the great sound sequences in films are "pov" sequences. The photography, the blocking of actors, the production design, art direction, editing, and dialogue have been set up such that we, the audience, are experiencing the action more or less through the point of view of one, or more, of the characters in the sequence. Since what we see and hear is being filtered through their consciousness, what they hear can give us lots of information about who they are and what they are feeling.”

With the advances in audio middleware, and the imminent arrival of the next generation of consoles, we are in a position to flex some fairly serious creative muscles in all areas of game development. The new FMOD Studio is shaping up to offer amazing creative potential with incredibly flexible mixer snapshots allowing sound designers control over these functions with little programmer support required. The ability to create generative audio looks to be getting supercharged in the way sounds can be broken down into the disparate elements and recombined in series to create huge variations in sounds. Wwise appears to be heading in equally powerfuls directions, and there is no doubt that the big in house audio middlewares will quickly catch up if they haven’t already.

Even with the current trend for Auteur games, the development of games is inherently diplomatic, and requires the skills and expertise of all departments. They are iterative, making experimentation more possible. They now include all departments in their pre-production and their design phases, and if game devs aren’t involving everyone, they will be left behind. These factors, combined with the epoch we are currently entering in games and popular culture, is precisely what excites me right now.



by Michael Theiler
Sound Designer & Studio Director at Kpow Audio