Book Review: Real-Time Cinematography for Games

The movie biz and game industry have a lot in common these days, with developers increasingly drawing on the lessons of Hollywood to bring a cinematic element to their games. In that vein comes Real-Time Cinematography for Games, by game developer and computer scientist Brian Hawkins, which translates the time-tested principles of live-action filmmaking into the language of interactive gaming.

In Hawkins' cinematography model – which mimics the structure of a real-world film crew – the two most important elements of a cinematic engine are its cinematography and editing agents. These two, under the guidance of a director agent, make all aesthetic and technical decisions about a given scene, which they implement through a squad of secondary agents, such as a camera operator and a gaffer.

As Hawkins explains, shooting, lighting, and editing are deeply intertwined in a real-time game engine. The author thus addresses all of these topics as he presents his integral cinematic model, and divides the discussion between general creative principles and in-depth technical analyses of the various aspects of filmmaking.

The casual reader might lack the technical prowess necessary to fully understand the more mathematical sections of the book, but the creative discussions are accessible to readers at any level. What emerges, over the course of the text, is a remarkably complete technical overview of adding cinematic quality to real-time video games.

The Eye of the Camera

In the early parts of the book, Hawkins covers the camera side of the equation – camera position and movement; using pans, tilts, dollies, and zooms for cinematic effect; obeying the line of action, observing the rule of threes; and other rudimentary principles of cinematography. He also addresses more advanced topics, including lenses and filters, motion blur, and depth of field.

In each chapter a technical section presents mathematical models of the concepts covered, and discusses methods for implementing them in real-time.

After the camera chapters, Hawkins moves into lighting, where the topics include the three-point lighting setup, the use of contrast and shadow to convey emotional and conceptual subtleties, the role of practical lights, and other similarities between lighting in CG and live action. There's also some discussion of the many differences, such as the ability in CG to activate shadowing and reflectivity on a per-object basis.

Rounding out the lighting section is a useful discussion of programmable shaders, and a few pages on the use of color in film and games, complete with full-color photo plates.

Assembling the Cut

On the other side of the equation is the editor agent, who is responsible for selecting and sequencing shots into a cohesive scene. In real-time cinematography, this process happens at the same time as the camera work, and thus the editor agent plays a particularly crucial role, responsible for both aesthetic presentation and storytelling quality of a scene.

The creative principles in the editing sections range from simple technical rules (e.g. the “30 degree rule,” dictating the change in camera angle between two shots), to the complex interplay of contrast and affinity that helps an editor determine how to prioritize the goals of a scene. These include aesthetic goals – such as color balance and framing – as well as story goals, such as what mood to convey, and what objects or characters to emphasize in frame.

The editor agent is also responsible for a number of “post-production” processes – splicing dialogue, creating transition effects, inserting overlays, and using real-time compositing to handle complex shot setups. These functions all tie back to the wider set of algorithms established throughout the book, and round out the overall discussion of real-time editing.

Hawkins spends the last sections the book discussing the role of sound in cinematography, including the mood-enhancing power of Foley effects, computer-generated sounds, pre-recorded sound clips, and a musical score – all handled by sound agents dedicated to their specific tasks. He also devotes a short section to the relationship between the director agent and the game designers themselves, who are of course ultimately responsible for the sum experience of a game.

Real-Time Storytelling

The take-home lesson here is that shooting, editing, and creating sounds are inseparable functions in the real-time world, since they all occur simultaneously. By modeling the separation of tasks on the structure of a real film crew, Hawkins achieves maximum control over his in-engine cinematic architecture.

But for all the extended metaphor of the book, Hawkins is well aware of the differences between real-time cinematography and traditional filmmaking. In some ways, he notes, computer animation improves upon live-action filmmaking – for instance, by giving artists the ability to simply hide objects intruding on a clean shot, or to create complex camera moves that might be impossible in the real world. At the same time, Hawkins notes the limitations – such as the complexity of calculating per-frame motion blur, which in film occurs naturally.

More generally, it's worth noting that the player is king in Hawkins' approach to real-time cinematography for games. He reminds us frequently that the aesthetic elements of a scene – no matter how well calculated – are only useful if they enhance the player's experience, especially in terms of story, and takes care to work storytelling goals into his overall equation.

Integral Cinematography

The strength of Real-Time Cinematography for Games is in Hawkins' ability to blend creative principles with technical know-how, translating the time-tested rules of the camera into equations and logical constructs. Combined with the creative discussions, and supported by a useful (though rarely mentioned) companion website, this approach makes the book accessible to both beginning and advanced readers.

The sum total is a comprehensive comparison between creating movies via a conventional filmmaking pipeline versus a real-time animated environment. Through a combination of foundational creative principles and in-depth technical explorations, Hawkins synthesizes these two approaches into a decidedly integral model of real-time cinematography for games.

______________________________________________________