Photorealism In Video Games - A Worthy Goal

Cross-posted from my blog at http://livewareblog.wordpress.com/

Capture

A response to Errant Signal - Photorealism

In his video Campster details why we should not be chasing photorealism in video games, I explain why we should.

Pipe Dreams

I have to chuckle every time someone uses this term in context to anything technological. Not because the tem amuses me in itself (which it does, considering the context and etymology), but because it reminds me how history tends to not only repeat itself, but also how people still lack the imagination to extend their way of thinking past the here and now. I will quote the first of Clarke's three laws that has held true over the decades:

What baffles me is that Campster, a man of certain respectable age (i.e. older than me), sees Photorealism as a pipe dream, unachievable (or so fantastical and uneconomical as to be almost unachievable in our lifetime) while he has lived through the amazing technological revolution of video games in the past 20 years or so.

Ultima Underworld - The Stygian Abyss, the first ever full 3D first person RPG, 1992 (according to Wikipedia)

28586The_Witcher_2_Screenshot_18

Witcher 2, most advanced graphics in an RPG as of 2012, sets computers on fire @Ultra

What can we expect in 2032?

Can you really, honestly, say that photorealism is unobtainable or uneconomical? A Witcher 2 level of fidelity was unobtainable and uneconomical in 1992, the level of fidelity of 2032 is unobtainable and uneconomical in 2012.

Campster brings up the hurdles that need to be taken: realistic mesh deformation, detailed animation, hair simulation, realistic cloth and physics interaction. They are all true, they are all hurdles that are hard to achieve, even harder to put together into a coherent whole, but claiming them to be inherently unreachable?

Inklings of solving those problems already are beginning to take shape:

Hair simulation:

[youtube=http://www.youtube.com/watch?v=cpdRfncGa1c]

Real-Time Eulerian Water Simulation:

[youtube=http://www.youtube.com/watch?v=Jl54WZtm0QE]

Real-Time Soft-Body deformation:

[youtube=http://www.youtube.com/watch?v=9JPDtDfxf6w]

Organically Grown Procedural Animation and Interaction:

[youtube=http://www.youtube.com/watch?v=ySRvKzZsDqw]

Many more are working on other parts of photorealism.

Why, why are they all doing this? Do they want to produce the next Modern Warfare? Do they want to "pollute" our games with more brown filters?! Monsters!

Progress

Wow this article certainly is Clarke-heavy.

The people working on photorealism are working on it because its the way forward, its progress.

You couldn't have your Windwakers without dynamic shadows/lighting and depth of field/motion blur post processing effects, technology that was not designed for Windwaker, or artistically expressive games, but for photorealism and photorealistic games.

Developing techniques and solutions for 3D graphics is time and resource intensive, no singular artist, designer or developer can afford to do the RnD themselves. Not even the large publishers (except maybe Microsoft, Sony and Nintendo, hybrid technology companies) can afford to run their own graphics RnD department.

Every single one of the effects you showcase as being not photo-realistic, were developed because of photorealism. We take our ideas from nature, from photorealism. Neon glow effects and styles did not come from the ingenuity of the artist, they came from neon lights we see in real life.

You don't see the old masters paint in the style of Tron, because the real life counterpart did not exist, the inspiration that made the style happen.

Photorealism is the single, most effective, driving factor for developing new technology and artistic games are benefiting directly from the solutions this research provides.

Video Game graphics are at the stage of medieval art, we are still trying to figure out our techniques for expression, finding the right tools and the right solutions. Only if we achieve photorealism can we move past it.

Abstract, expressionism, impressionism, Cubism, surreal, etc. art-styles only emerged after painting was firmly grounded in realism (or at least as close as the technology of a paintbrush can get to realism).

23638-Madonna_de_Santa_Trinita_Cimabue_detalle_large

Medieval Uncanny Valley, Madonna de Santa Trinitá, Byzantium, 1285

Realism, 1854, Courbet

Photorealism, 2007, Baeder

Authenticity

No matter how artistically expressive your game is, our brains are wired a certain way, they see reality every day and expect things to behave in exactly (or approximately) the same way in simulations.

This is why we can tell that 3D graphics look "fake" and its the reason the Uncanny Valley exists. However, we can only take out certain aspects until something becomes less then simplified and becomes "fake" as well.

It has mostly to do with shadows, proportions, lighting and perspective, those are the things that our brain has an, lets say, intuitive understanding off and can immediately tell from fiction.

Color is far less problematic it seems, we still recognize black and white film as depicting photorealistic characters, and we don't feel its somehow "off". The problem starts with tonal changes and fake coloring when our brains reject the image as photorealistic.

jenna

Still photorealistic, tones are preserved, light and shadow is realistic. (c)Jenna Herman

jenna2

Jenna Hermans photo fucked around with in photoshop. The face loses its recognizable definition. Uncanny Valley.

3D graphics technology strives to be authentic, be it in Windwaker, Torchlight, or World of Warcraft.

Light casts shadows realistically, water reflects the environment, the perspective is realistic, the world is believable enough to our brain to not reject it outright but fantastical enough to entice us.

The most persuasive argument are the Toy Story movies. The enriched, higher fidelity and -physically realistic- lighting and shader models made Toy Story 3 the experience that it is and could not have been made with the distinctly less realistic technology of 1995.

Especially if you look at the fire-effects in the furnace at the ending, the dramatic physically authentic lighting with scattering, multiple dynamic sources, displacement and normal shaders. All this could not have been done unless someone worked on achieving the photorealistic effects in 3D graphics.

Subsurface Scattering and softbody meshes would not be present if it wasn't for trying to make realistic skin (Shrek), refraction, caustics and photon scattering wouldn't exist if someone didn't want to simulate realistic water (Finding Nemo), realistic fur and hair simulation wouldn't exist if it wasn't for the desire to render authentic animals. etc.

All those techniques would not be here if photorealism didn't motivate them to be created.

To ask to not pursue photorealism, is to ask to not have progress. Its to limit yourself to what we know and never take a step further into the unknown.

I do not advocate -only- photorealistic games, but the last time I checked, we didn't have a problem with stylistically and artistically diverse games in the marketplace.

Sure, not very often from tripple-A publishers, but can you blame them? Tripple-A titles are the Blockbusters of the industry, they cater to what the majority of the public wants, and the majority of the public doesn't watch experimental cinema, so why would they want to play it?