Videogame Aesthetics: The Future!

Depending on your point of view, photo-realism is either a scourge or a grail. The drive for it has come to dominate the visual aesthetics of videogames, and current technology seems to be pushing us very close to a peak. Limited markets and rising development costs, however, seem to indicate a gaping abyss. So will photo-real games be well crafted marvels of technology, or feats of economic hubris infused with mediocre gameplay? Will they be the ferryman to conduct yet more development studios across the Styx?

The photo-real push is obviously important to many people within and surrounding the game industry, as demonstrated not only by the persistent trend in commercial development, but also by work such as the System Shock 2 mod Rebirth, which replaced some of the models with curvier versions, designed for more powerful machines than the original game.

Yet increasingly, the push is sneered at. Among some of the gamers I know, the latest graphical offerings get little more than apathy. Critics cite rising development costs and the potential of different artistic goals, and are generally scornful of industry resources being poured into visually superior concrete and monsters. Nonetheless, they seem dangerously close to drowning under the effusion of marketing departments and most players.

In what appears to be a fit of turnabout, gamers often murmur "Of course, it's all about gameplay" when graphics blunder oafishly into the conversation. Well of course, interactivity is more fundamental to the medium than most if not all other parts of it. We'll always stand by gameplay: but it's graphics that will be handcuffing us to the bed during our next "business trip."

The industry and the market are bewitched by the idea of more pixels and polys. Higher visual quality is fair enough, but why is it equated with better stabs at photo-realism? What's the point of aesthetics at all? If they don't matter, how come E3 can sucker-smack a "wow" or two out of so many gamers each year? Why, after gushing over how good stuff looks, do we hypocritically trot out that almost apologetic load of bollocks about gameplay moments later?

I'm guilty of it. I think it's time that particular conversational old dog was taken out back and shot. By no means am I suggesting that aesthetics are the very substance of games, but obviously, "it" is not all about gameplay. I suspect even the most fanatical ludologists have been watching tech demos with the curtains drawn.

We may not know a great deal about what they are or exactly what they do, but aesthetics are clearly important to us. As a phenomenon, aesthetics have manifested in every culture and sub-culture throughout history, and furthermore survived the demise of each. From food through to music and architecture, all of our possessions and many of our experiences are purposely shaped by designers for aesthetic as well as functional purposes. Aesthetics pervade all media, and games are no exception.

The general value of aesthetics is not derived from any one particular style, as evinced by the massive variety in historical and contemporary design. So why do games seem to focus so singularly on photo-realism? Could it be that, because games lend themselves to simulation of reality, their aesthetics meekly follow? Are designers choosing a default option at the expense of aesthetic variety and potential?

Join me for this not entirely thorough survey of the visual aesthetics of videogames.

Fantasy & SF Affliction

So what's going on at the moment? In addition to gritty real world settings, games suffer from a lot of default Fantasy and SF imagery. There have also been many 3D cartoons around for the past decade or so in the form of Crash Bandicoot, Spyro, et al.

This seems to copy the aesthetics on show in a large portion of other screen based media, and maybe this could be instrumental in popularising the relatively new cultural form of games. However, are games really limited to "video" and "cartoons"? Pared down to just those two categories, games may seem tragically limited when compared to the aesthetic variety of other visual media.

Easily said. Trawling my memories of film for aesthetic variety is a somewhat barren endeavour, and it seems like a worrying omen for game aesthetics that I have to ferret around outside of mainstream cinema. Rotoscoped orks in the 1978 Lord of the Rings certainly weren't a high point, though the technique was used to much better effect in Waking Life, where live action was not only made to look like animation, but also actively distorted and enhanced by it. Films such as Run Lola Run and Amelie exhibit warped, surreal colour palettes that would translate easily into current game worlds. Sin City was a significant and radical modification of the Film Noir aesthetic, and furthermore was expertly translated to film from the comics of Frank Miller. It's in that field that we can find significantly more aesthetic variety.

The work of comics creators shows massive diversity: Jim Woodring, Tom Gauld, Chris Ware, Paul Pope, Peter Kuper, Mary Fleener, Robert Crumb, and Kyle Baker differ greatly in terms of colour, line, effects, and viewing angles. They represent a fraction of the variety in comics, which in turn are a fraction of the print media surrounding us. Within that vastness is found a staggering array of visual aesthetics, most of which can translate fairly comfortably into 2D games. Good examples are Orisinal, N, and De-Animator.

Of course, though a graphic design tool such as flash easily lends itself to aesthetic experimentation, the potential illustrated by print translates into any 2D game, for instance Project Rub, Spheres of Chaos, and Vib Ribbon.

Clearly, games do not preclude aesthetic variety. Furthermore, the aesthetics of games are not merely to do with HUD and menu graphics, but are about the way in which game worlds are presented. There's a lot to explore, as we're no more aesthetically limited to photo-realism than we are bound to simulate realistic processes with our game mechanics.

So why is there no renaissance of imagery in computer games? The kind of games shown above seem to be a distinct minority in comparison to War FPS Iteration Y.

With regard to 3D game aesthetics, the strongest traditional media connection is sculpture, which has luckily become jam packed with aesthetic variety in the last century or so. Michelangelo may have produced fantastically realistic marbles, but even a cursory glance into the recent annals of sculpture reveals the work of Henry Moore, Barbara Hepworth, Andy Goldsworthy, Joan Miró... the list spirals outward, widening with it the staggering aesthetic potential of three dimensional craft and representation.

Take just one example: Marino Marini, who, among other things, was known for sculpting young men riding horses. Some are fairly traditional representations, somewhat realistic and heroic, but his later work on the same subject developed into nightmarish and decidedly abstracted forms. Far from "realistic", nonetheless still recognisable as horses and riders.

The tools a sculptor or painter uses to make abstracted work can be the same ones with which they might strive for realism. The same is true of 2D and 3D game content production tools: they are just as usable to produce abstract and symbolic content as they are for photo-real work, perhaps even more so.

Taking a historical perspective though, could the peak of realism in any form of media also be a plateau that acts as precursor to wider experimentation? The push towards "realism" is visible in many forms. Film has climbed from silent, grainy, low resolution black and white to high definition imagery accompanied by surround sound. Sculpture and painting both emerged from rough neolithic beginnings to the eventual high fidelity representations of the Renaissance and following periods. Similarly, because in the beginning hardware limited representations to simplistic abstractions, the pixelated sprites of early game design are the equivalent of cave paintings.

So how is our progress toward photo-realism? We obviously aren't there yet and won't be for some time to come. Take anything that's currently at the leading edge of the photo-real push, such as Project Offset or Unreal Engine 3. You'll certainly find some very pretty and visually impressive stuff, but it's still not quite good enough to dupe. Despite claims of cinematic quality, you can still see polygonal outlines on models, if you look. Photo-realism will have been achieved when, as a photographer and level designer, I can swap those two parts of my portfolio and actually fool people.

All 3D games that have so far been a part of the photo-real push are actually cartoons of an oddly lit, particularly angular style. That style could always have morphed in various directions, yet at each iteration of Moore's Law, developers threw what they had as far toward realism as they could.

The photo-real push is almost as established a part of game culture as shooting or driving, and for some it is becoming just as tired. Maybe though, games have to push all the way to photo-realism before intentionally pushing away from it becomes more than a marginal pursuit.

Tool Use

Perhaps a thought tool can be of assistance. The comics creator and analyst Scott McCloud devised an extremely useful one, named the picture plane, for describing visual representations. Though applied explicitly to comics in his book "Understanding Comics", as he explains it "represents the total pictorial vocabulary of comics or of any of the visual arts". A game related equivalent is shown below.

You can read an explanation of the plane by Scott himself, here. Basically, the bottom left is the extreme of absolute photo-realism. In the bottom right are things that are somewhat abstract and semantically meaningful, and at the top is pure abstraction; line and form without any specific symbolic meaning. Games are as easily placed within it as any other visual media, though the development of photo-realism means that if thoroughly charted, games would largely follow a chronological progression from right to left. Being possessed of limitless artistic potential, rules of aethetics such as harmony, rhythm, pattern, contrast, chaos and structure can operate at any point on that plane. The photo-real push, though, seems to have inherited its rules of beauty from chocolate boxes and post-apocalyptic fiction, the net result being that the latest gun renders are as jaded and ubiquitous as any camera club photographs of trains or waterfalls.

While developers and publishers race to cram themselves into the bottom left corner of Scott's plane, and are squeezed dry of all the resultant ugly hype, some will continue to work in the largely unexplored territories of visual representation. It's those implementations we need to push game aesthetics outward, but what's been done so far?

Current and Recent Work

Quake NPR:

A mod for the original Quake, this replaces the original 3D rendering with various non-photo-realistic visual styles, including pencil sketches and blueprints. All the files necessary to run it can be downloaded from the website.

Rez:

A modernised low-poly and wireframe aesthetic inspired by Kandinsky. The visuals and sound effects are synchronised to the music, making it utterly hypnotic.

Darwinia:

An intentionally polygonal aesthetic is used here to create a bleak landscape representing a mainframe, which is lathered in pixel-like textures and sprites.

Beyond Good and Evil:

This might seem an unusual choice, as it has a fairly typical aesthetic somewhere between photo-real and cartoony. However, blur effects are used several times throughout the game to convey changes in the consciousness and emotional state of the heroine, as opposed to their usual use in conveying world events such as supernatural occurrences or scenery whipping by a speeding vehicle.

Shadow of the Colossus:

Currently in production, this uses 3D rendering with a decidedly non-photographic colour palette, often dominated by green and possessing much more harmony than colour contrast.

Hollow Moon:

A UT2004 mod set on the moon, in which the colour palette is restricted entirely to black and white. Sound effects are intentionally omitted, adding to the sense of isolation. The first-person view is mediated by a grimy visor, while the view from vehicles bears a cross pattern reminiscent of Apollo mission photographs.

Evil Genius:

A retail game with a deceptively spartan visual aesthetic based on 1960's spy thrillers. Everything is bright, 3D, and minutely yet minimally detailed. This pared down aesthetic serves to accentuate the peerlessly animated myriad actions of your villainous lackeys, such as interrogating enemies, and stealing gold from you then hiding it.

Katamari Damacy:

The designers of this faced a problem, in that they needed to populate a scaling world with many hundreds of objects. PS2 poly budgets meant that they could only use simple 3D objects, and this limitation was turned into a strength, infusing the bright, iconic, vaguely cubist aesthetic style at all levels. The aesthetic was retained and explored a bit further in the sequel with an undersea setting and a ratherwhimsical purple and pink flower garden. Also, the camera was improved in an aesthetically pleasing manner, insofar as any objects that come between the katamari and the camera have a transparency mask applied to them, the shape of which changes from level to level.

Arguably, Katamari Damacy is also a good example of the aesthetic being tied more deeply to the game. The simple, almost child-like visual style is a good complement to the basic game mechanics of rolling around collecting stuff, and also the childishly joyous and simple storyline.

Lego Star Wars:

Another example of aesthetic integration, this features not only the visually distinct aesthetic of lego, but also has the inseparable mechanics of lego meshed perfectly with the mechanics of Lucas' "Force", both in turn being rolled up into a very efficient and quite delightful game mechanic. The sparse lines of Star Wars sets and Lego sets complement each other well, the lego being constantly present and unselfconsciously blended with more detailed backdrops and world objects.

Mono:

Briefly making our way back to 2D games, this is another example of a game aesthetic being tied to mechanics. It looks somewhat like an Asteroids clone, and it is, but the objective is to paint the backdrop by bursting the primary coloured globules to change the colour values in the right places. The goal is to paint 100% of the screen, and if your ship collides with anything it creates a blank spot.

Okami:

Work under development, in which you play the Japanese Sun God Amaterasu incarnated as a wolf. The entire 3D game is rendered in the style of Japanese paintings and woodblock prints, making the aesthetic both stunning and thematically relevant.

Faddish Allure

Whether you equate "concept" in games with narrative or the ludic elements of gameplay, it's clearly possible for game aesthetics to have a deeper tie to the nature of a game. Despite that we use shinier visuals each year to neaten up yet more iterations of established concepts, aesthetic design is more than just caked layers of whorish make-up; though such unfavourable comparisons could be made of many franchises and genres.

None of the rendering methods and artistic idioms of the games above need be a one-off; in theory they are as reusable as photo-realism. However, retreading any such territory seems to earn condemnation as a novelty bandwagon hopper for anything that isn't trying to be photo-real or explicitly different. At least, such accusations were levelled at cel-shaded games in recent years. Soon after Jet Set Radio had emerged, there came a plethora: Viewtiful Joe, XIII, Zelda: Wind Waker, Sly Cooper, to name a few.

The glut had it derided as a faddish rendering technique, with anti-cel-shading rants and musings popping up. Such detraction could in part be because the addition of black lines to the outlines of 3D models is a purely aesthetic inclusion, whereas it's a necessary part of the physical production of real cel-animation. For example, compare any cel-shaded game with Out From Boneville, a comic adaptation where cartoon aesthetics are created purely by form and colour. 2D graphics, such as menu items, icons, cursors, speech bubbles, and eyebrows still have (or are) black outlines.

A similar retention of black lines on 2D art can be seen in Lego Star Wars. Bad Day LA also utilises black lines in 2D artwork, notably including the skins for 3D models. Nevertheless, despite black outlines being an entirely aesthetic addition to 3D games, current titles such as Killer 7 and Okami seem to indicate that cel-shading is not just a fad, nor, in the latter case, a cliché.

A possible explanation of the allure of photo-realism, as well as the tolerance of aesthetic similarity in offerings of that ilk, is yielded by an aside from Scott's picture plane. He points out text as the utmost iconic abstraction, because though writing was developed from pictographic representations, in modern languages a written word retains meaning while exhibiting none of a physical form it might denote.*

*p46 and 47, Understanding Comics.

This means that between text adventures and Tetris, games have already moved quite far towards the extremes opposite photo-realism, and those areas are now open for free roaming exploration within present technological limits. In contrast, the absolute photo-realism of the bottom left corner beckons to us as territory that we are currently denied. It's a clear goal, toward which each wave of technology takes us slightly closer. Being such a comprehensible objective, it is also inevitably more comfortable than pondering the relevance of subjective aesthetic choices.

So will photo-realism be a Garden of Eden? Yes and no, I think.

The Good, the Bad, and the Exquisitely Ugly

There are a number of problems and hopes with photo-realism.

Display Technology:

Though the technological limits of displays are gradually diminishing, their shortcomings are still significant. VDUs are simply not a comfortable thing to use, and incoming technologies such as electronic paper offer the prospect of electronic content with the comforts of print media.

Furthermore, even our largest current screen resolutions are significantly impoverished in terms of visual quality when compared to print, and while present technologies allow for the variety of it to cross into games, much higher display resolutions will allow for its richness to also be transferred.

For instance, they would make possible a Pointillist renderer, the likes of which would simply be unfeasible at lower resolutions. To extrapolate: higher screen resolutions backed up by more GPU power and RAM will not just enable higher poly counts and texture resolutions, but also more complex and detailed ways of rendering and seeing entire worlds.

Developmental Crises:

Art assets as we know them are expensive, with costs rising in each generation of hardware. This is a catastrophic obstacle that cannot be ignored, as it means that only the largest publishers will be able to afford photorealism with current production methods. The upward spiral of development costs, so far offset by a downward spiral in quality of life for some developers, means that the blinkered push for photo-realism using traditional management and production techniques seems to be the enemy of mental health, interpersonal relations, ethical conduct and the medium itself. If commercial game development is to make use of photo-real capabilities within reasonable monetary and ethical bounds, art will have to be licensed or made with smarter tools and methods. Will it be so expensive to develop actual photo-real games a decade after they are fresh?

Even if the rising financial hurdles of production can be destroyed, games may have an existential crisis to overcome. When actual photo-realistic games descend upon us*, will alternative modes of aesthetic design seem more credible to developers, publishers and players?

* Insert as applicable:
- "like a glowing musical spaceship"
- "from the malodorous viscera of the industry".

Uncanny Valley:

With regard to the rendering of humans, another problem that photo-realism may face soon is that of the Uncanny Valley. The roboticist Masahiro Mori asserted that near realistic humanoids are repellent to people. This might also be true of virtual representations.

However, the Uncanny Valley is a disputed hypothesis; and it doesn't seem to have been a major obstacle to film-makers using pre-rendered CGI. Even if true, it begs certain questions in relation to game design: Does it operate only when a near real humanoid is contrasted with real humans? How does a screen affect such perception? Does the negative response depend on the comparison of things in a single context, and if so would not the harmonious rendering of all entities in a game world make it irrelevant? If not, we might be forced to explore other aesthetic options until we can produce realistic imagery and animation with ease.

Freedom of Line:

As mentioned earlier, we have a way to go until we actually get to full 3D photo realism. For now, no matter how close our work is, everything we produce in games is to some degree abstract or iconic. Some have sought to circumvent this with games that are based on video footage or contain portions of it, such as The X-Files or some of the Myst series:

However, using film of live actors inevitably places limits on the amount of interactivity such a game or scene can have, as shown by the manifestation of the X-Files as a point and click adventure, and the FMV in Myst IV being, typically, relegated to cutscenes. Also, CGI imagery and photographic imagery tend to clash unless large amounts of money are thrown at them, as with film production. With game assets, such mixtures have tended to look conspicuous, though the gap is closing, as demonstrated below by Nick Bertke's HDRI renders of Half Life 2 characters on photographic backgrounds:

Though materials and lighting are currently moving very close to photo-real rendering, we don't have real-time photo-realistic geometry yet, simply because our hardware can't spew enough polygons. Artistically, this means that those producing assets for real-time 3D rendering just don't have the same freedom of line that an artist with a sketchbook has, or the freedom of form possessed by a sculptor. At their past and present levels, texture and poly budgets are an unavoidable hindrance to the aesthetic quality of games.

We've come a long way since the flint-carved figures of early 3D games, but there's still progress to make before we're producing the game equivalent of sixteenth century marbles. At that point, poly budgets will afford us much better opportunities to experiment than we are allowed by the present but rapidly shrinking necessity of abstraction.

So maybe the growing hostility toward photo-realism should instead be directed at the effect it is having on the development community. Though it makes for a myopic obsession when compared to the vastness of the picture plane, photo-realism is nonetheless a worthwhile technological achievement to aim for, because it is through this that games will attain the sensation of a lucid dream. Furthermore, far from being an eventual dead end, the technology behind visual realism could actually be an avenue that opens up other interesting aesthetics by creating emergent possibilities at all points on the picture plane. While it obviously isn't the only route for discovery, many technologies developed in the cause of photo-real rendering, such as HDR lighting and normal mapping, could be fed back into abstraction and iconography to produce effects that are both subjective and surreal.

To an extent this is already happening in some titles, where current level photo-real worlds and characters are distorted in decidedly non-photographic ways. For instance, in F.E.A.R. refraction is used for entirely mundane simulations of glass, but it is also applied to explosions and bullet trails to create cinematic effects:

Conclusion

Much like the rest of the medium, game art is a maturing phenomenon; the aesthetic variety shown here is a fraction of what is and will be open for exploration. Thanks to Scott's Triangle, we have the map. Technological development does not solely equate to the development of photo-realism. While the aesthetic development of games will to some extent occur in conjunction with improved technology, and while the corporate games industry will continue the photo-real push, what's already been done shows that utilisation of newer hardware is by no means the only boundary to be pushed. Though commentary, management and markets may have an effect on the aesthetics and culture of games just as they do any other form of media, discovery will remain squarely in the hands of artists and programmers. Among students, professional employees, indie developers and mod-makers, there is a minority out there experimenting, as people have been doing with computing and visualisation for decades.

The saleability of such fruits in game form is a complex and erratic proposition, but it seems scarcely relevant. Such exploration is surely an interesting objective in itself because aesthetics are inevitably a part of this medium, and as such transcend any industry trends or catastrophes. Theoretically games could host a period of aesthetic experimentation comparable, at least in visual diversity, to the artistic movements of the last century. Perhaps a quixotic hope, but I think it's a worthy one.