Technology matters in an interactive digital medium

[This post can also be found on my website.]

Sony's PlayStation 4 was announced yesterday. Throughout the myriad conversations being had in regards to its reveal, one topic boils under the surface: whether we need better technology. Mainly this is usually framed within the context of David Cage's segment of the Sony event in which he equated the state of games now to the state of silent film. In the silent era, he posited, emotions were hard to get across as sound and proper editing were not yet part of film's language. As the technology got better, film could use the new techniques available to be more subtle and engaging, especially in regards to acting; broad pantomime and gesticulating were no longer a necessity. In a pithier statement, he was saying that better technology in games (framed in the way of higher polygon count) would allow for more emotion. (His speech can be viewed here.)

Some take issue with this. Some say that there are plenty of emotionally resonant silent films, and that polygon count is not required for emotional connections. These are true statements. But both miss the meaning of Cage's words: that technology in video games is important. He may not have framed the point in the best way, but he's admitted in the past that, for these kind of speeches, he feels he "need[s] to be a little bit bold" to rile passions.

For the statement that silent films don't need better tech to connect with audiences, they are missing the larger scope. Yes, you can find emotionally resonant silent films, but that doesn't mean all films need to be silent. If all films were silent, we would be missing out on some of the best films ever made that employ sound (for example, Casablanca), and further still, those that employ colour (for example, 2001: A Space Odyssey). Sure, we could probably make close facsimiles if we limited ourselves to technology from the silent era, but we would be losing so much in the process.

For the statement that polygon count is not required for emotional connection, again, the larger point is being evaded. As Cage demonstrated with his tech-demo of an elderly man, subtlety requires greater detail, especially if one strives for photoreal (or near-photoreal). Yes, you can get a similar emotion across from simpler shapes, but you would also lose subtle details like the wrinkling of features. With subtlety, you are not forced to pantomime.

But neither of these points truly capture why better technology is important. It's a common fallacy that better tech only means better visuals. Not true. Better tech means whatever a developer wants it to mean. Games are very much tied with real-time simulation; it's a main reason why our medium is unique. Want an open-world game with various AI controlled actors behaving in a believable matter with multiple outcomes? Great. But you're going to need technology to pull it off, and pull it off in real-time. The real-time factor is what's crucial. Film can do amazing similar things, but each frame will need hours or days to render in renderfarms. Add audience interaction on top of that, and the work must be able to react to outside input. None of this is technologically cheap.

"A bad workman blames his tools," is an oft-cited proverb, but it's not completely accurate in this case. For example, in the realm of painting, proper lighting was a matter of discovery for the artists and learning how to "see". Once they understood how light bounced, reflected, and illuminated, they could take their time and create works with believable lighting; the tools were secondary. In the digital realm, however, "lighting" in any real sense must be simulated; baked lightmaps betray the interactivity we strive for in games. With digital simulation, discovery of how it works is not the prime issue; we know how light behaves in physics, and physics can be translated to code. The issue is the technological power required to actually run a simulation of believable lighting. With video games, the additional issue is doing all this for each frame at, ideally, 30-60 times per second. You want to make a work that conveys emotion via its use of subtle lighting? Great. But do your tools even allow you to do the job in the first place?

Another oft cited proverb is that "creativity requires constraints". I would agree with this. But some ideas cannot be properly expressed under certain limitations. Do all works need the greater overhead that better technology provides? Of course not. But in the arts, you want more variety, not less. Restrictions should, ideally, be self-imposed based on personal decisions or imposed by the need of the concept, not by the technology itself. Better technology in the arts helps us all. After all, video games require complex machines with video rendering, audio creation, and input/output to even be experienced in the first place; they require, in the full sense of the word, computers. On this matter, we are not like the media that came before us. On this matter, we are unique, like it or not.