For some time now there has been a trend in narrative-based console games towards "cinematization" -- or put simply, "sounding like film". What exactly does sounding "as good as film" mean? Is it simply technically being able to do the same things that film does at the same resolution? Or are there more creative and collaborative elements involved?

Next Gen Audio: Price of Entry Features

In the previous generation of consoles (PS2, Gamecube, and Xbox), memory had been sparse enough to impact on the sample rates, or "perceived quality" of the audio content. Games which had a large amount of audio content, such as Grand Theft Auto: San Andreas on the PS2, had to make compromises on the quality of their ADPCM audio assets in the game, often resulting in scratchy-sounding assets, such as dialogue.

It is certainly the case now, with advancements in audio compression, particularly with the adoption of MP3, Ogg Vorbis, XMA et al, that sample rates can now rival those of a film soundtrack (depending on the amount of content required) on the PS3 and Xbox 360.

To add to this, with the limited RAM sound memory of previous-generation consoles, the amount of sounds that could physically be loaded and played at any one time was strictly limited. It is also now the case that more sounds can be played simultaneously than they could on previous generation consoles (more voices available) and around ten times more sounds can be loaded into RAM.

Of course there is also the addition of software DSP such as reverbs and high/low-pass filters that can process sounds and be tweaked at run-time, rather than relying on the crude reverbs that shipped with the development kits on previous-generation machines, and made permanently baked-in effects processing on sound files a necessity.

With more sophisticated mixing and post-production techniques (not necessarily next-gen exclusive) now being used in the development of sound for cinematic game titles (as seen recently with the dedicated post-production audio time applied to Scarface: The World Is Yours and more recently with the mixing time afforded in post-production to Heavenly Sword) there is also the opportunity to use and tune sound in such a way that it can focus attention very clearly on particular sounds at particular moments in gameplay. This is done by reducing, or removing unnecessary sounds at any given moment moment using real-time interactive mixers and DSP effects.

Heavenly Sword was given post-production audio time in order to do a full in-game mix

So these are the "price of entry" features in next-gen audio development, but where can innovation and improvements be made?

Designing A Game For Sound

In essence, it appears that game sound designers finally have all the qualitative tools that film sound designers have been using for many years. We have already seen many fine examples of technically improved game audio on the Xbox 360 such as Gears of War, BioShock and MotorStorm all supporting these "price of entry" features of next-gen audio development.

However, this is where games, and similarly many motion pictures, hit a proverbial brick wall in terms of sound -- because story, and more pertinently gameplay, must be designed for sound from the ground-up. There is an aesthetic/collaborative issue at the heart of being more like film, not a technical qualitative issue, and it is something that arguably game sound is agile enough to do better than film sound.

The movie sound designer Randy Thom has very lyrically stated that a movie must be designed for sound, rather than the other way around. This essentially means that a sound designer, or a director who cares and allows opportunities for sound to be used well in a film, should be involved as early as possible (meaning the pre-production period) in helping craft the storytelling elements of the movie.

Randy's work with director Robert Zemeckis bears testament to the benefits of this collaborative process with movies such as Cast Away and the brand new Beowulf containing many fascinating "sound moments".

Although still frustratingly uncommon among movies, game sound can certainly learn, and improve by using this practice as there are often full-time, in-house sound personnel physically sitting in the building at the time of pre-production.

Another advantage in game sound is that the game development process, and the roles of game development staff, are more mercurial than those working in the movie development process - the motion picture industry, particularly at the Hollywood level, has solidified into very rigidly defined jobs at very specific times whereas the game development model is still so young, experimental and flexible in terms of structure that it gives a good chance of getting this right before the development process solidifies.

There prevails an attitude among game designers and producers (much like the attitude of movie directors) that sound's job is to work miracles quickly and cheaply at the end of production with little or no collaboration in the creative process up to that point. One such core principle in designing a movie for sound is allowing the characters, at the script stage of development, a chance to listen to their environment and to hear things, essentially giving the characters "ears". This allows the sound designer to exploit more creative opportunities using the point of view of a character later on in production and post-production.

Narrative games certainly have strong characters, and many of them often have very identifiable points of view. It is a real challenge for sound designers to become involved and be more influential in the early storyboarding and early game design concepts. In many ways the potential for point of view is much stronger in games than in movies.

The taunting mechanic in Scarface: The World is Yours is one such example of where a sound feature is used to both further the gameplay from moment to moment and also to re-enforce the way the player experiences the character of Tony Montana. Dialogue is used here to both express the rage of the character during combat and also as a method of "building up points" to activate "rage mode" -- itself a core gameplay feature.

I believe that in order to change the trend, it is up to us -- as sound personnel responsible for making great games -- to be the ones who push for further involvement in pre-production on game titles. After all, it is not just going to happen on its own, overnight.

There are now so many ways of manipulating sound available to game sound designers, all they need is a game that really welcomes and relies on sound to form a cornerstone of its gameplay. There certainly isn’t the game equivalent of a movie like Apocalypse Now out there right now, which was a movie that arguably allowed the most opportunities of any 20th century film for sound to come in and tell at least half of the story. But there could be.

One of the biggest areas of challenge for audio on next generation cinematic titles, and indeed for all development disciplines, is to become much more interwoven with one another from the earliest planning stages of game design both technically and aesthetically.

Proactive Sound Design

For audio personnel, pre-production is typically a very quiet time on a video game project -- perhaps being involved in some "look and feel" movie material for greenlight meetings is the extent of the workload.

However, getting involved in the game design work is fundamental to integrating audio into a game. Try to re-conceive of this as your most productive and collaborative time on a project, a time where lots of different ideas can be tried out, a very experimental period. There are several ways that audio can be brought into the game development process at an earlier stage:

Ensure that sound is clearly represented at all the pre-production game design meetings, particularly early story concepts. Think about how sound will change over the course of the game.

Ensure sound is present at all early technical meetings in terms of art pipelines such as animation. Sound should be considered technically as an integral part of any art asset or any animation pipeline. In dialogue, consider driving character animation from sound, rather than having to retro-fit sounds to match animations, which is frankly impossible in terms of dialogue content.

The direction for how a character looks, sounds and feels should be a collaborative direction invested in by everyone involved in its creation, so being involved in concept art for major characters is also a great place to get ideas moving.

Be present at storyboard meetings in order to have input on how scenes play out, work closely with the DP in allowing opportunities for dialogue to feel natural and also allow for the characters to hear things and interact with sounds within cinematic scenes.

Be in as many game design meetings as possible. In essence, at this stage it would be most beneficial to be considered part of the design team by the designers. Consider sound as part of any gameplay device or feature, and how it functions. Always allow for, and state the case for, sound to lead the gameplay.

Be proactive! Get out of the studio. Remember it is very rare for game designers, animation artists or others to come to you looking for your input on the design of a feature. They usually only find your studio when they need something from you, i.e. when they have already finished the art or design and now need you to make it sound good. Take a laptop and sit among the game designers or artists for a few weeks, be visible on the team, become a familiar part of their production environment.

Foster a collaborative environment. Most importantly, remember games are not there to satiate the sound designer's ego! Great sound cannot save a terrible game, so being mutually accountable for the work of the rest of the team is critical.

As a sound artist you are collaborating with producers, artists, coders and designers in getting what is best for the game, so be prepared to listen to others' opinions and try out their ideas, as well as expecting others to listen to yours. Next generation offers the developer nothing if not the chance to work with greater awareness of other disciplines.