From Cave Painting to CRYENGINE

Note: This is an extremely boring/in-depth version of the speech I ALMOST gave at the Crytek booth during GDC 2015, but then wimped out and gave a tech demo instead. Reposted from my blog: www.neon-serpent.com
 

Riding a flaming giraffe and firing a stream of Nutella into a homeless shelter while wearing a Speedo made of food-stamps: something that technically hasn't happened yet (in the name of Art) but probably will soon.  The capital-A Art World is all over the map these days in terms of finding new (often pointless) boundaries to cross, but it took them forever and a day to get there.  Once upon a time Art was very rigidly structured; innovation was slow and consistently met by critics saying, “it's shit,” to masterpieces that weren't part of the narrowly accepted parameters of the era.  Over time each medium found it's own path to Artistic freedom; some paths have been more direct than others, but each path has been unique...OR HAVE THEY? (fake drama)

Take the visual history of video games, for example, it reflects a mirror image of the advancements in one of our most respected, time-honored art-forms: painting.  Their respective paths from abstraction, to realism, to abstraction again are nearly identical.  In both cases, it was technology and invention that led these mediums along their visual journey.  This reflection of styles wasn't intentional on the part of video games – it was truly a coincidence brought on by limitations – which makes this observation all the more interesting.  Out of pure happenstance all of the major generations of gaming technology seem to match up perfectly with the great movements in painting.  Not unlike the X-Men, it's uncanny.  And since everything needs a catchy name these days, I'm calling this trend, “The Swordbill Effect,” because it sounds cool and stuff.

Note:  Before I go any further, I want you to know this is not scholarly work.  I took a bunch of Art History classes back in University, scribbled some illegible notes, and moved on with my life.  That's it, that's my credentials.  So please, don't take this as fact, just as observation.  Legit art scholars, feel free to do some real research into the subject if it interests you.

Here we see time represented on the horizontal axis, with the left side representing the cave paintings of prehistory, (or the first video games), and the right side representing contemporary art/games of today.  The vertical axis represents the degree of realism seen in our work, with the bottom representing complete abstraction, and the top representing reality itself. 

Now that my perfect infographic has made it crystal clear you'll see our slow but steady rise towards realism.  This is followed by a divide where differing, non-realistic styles start to appear.  After that a critical event occurs that sends us racing back towards abstraction, while also helping us master realism.  From there, once we've hit both extremes, we see a spectrum of all types of Art, which is where we are now, the, “everything goes,” era.  

Anyway, I called it Swordbill because of it's shape, check it:

See what I mean, fairly close right?  I also stumbled across this when looking up Sword-Billed Hummingbirds:

Fact of Life: Sword-Bills will stab other hummingbirds in the throat!  Of course they do. I know if my nose was a sword I'd be morally obliged to do the same thing.  But let's not get distracted.

So far this probably all seems like gibberish, so let me break down into waaaaay more detail than anybody cares about, (the art history stuff, not the throat-stabbing stuff).  By the end you'll either be totally convinced or totally bored; 50/50 chance.  So let's start by going section to section; we'll start with The Slow Ramp

THE SLOW RAMP

Yeah that's right, I named the sections.  So what?

This is the long stretch of history that sees us slowly working towards the mimicry of reality.  Advancements come at a snails pace, (especially when it comes to painting), but each brings noticeable upgrades in fidelity.  But it's much easier to just show you what I mean.  Let's history!

Note: COL=color, PER=perspective, SHA=shading (depiction of light)

Here we have the start of both mediums.  Each had access to extremely limited palettes which were used to create very basic lines and shapes.  Our cave painting ancestors had no artistic conception of light, perspective and so on, whereas of course we knew of all those things when we started creating games.  In our case, we were limited by technology; despite our full knowledge of Art we couldn't bring that across in our graphics...yet.  We were working within the very tight constraints of early computers, and it showed.

Note: REP=representation of figures/objects

Similarly, in both cases our representation of humans were simple silhouettes: just enough information to convey what the artist intended and absolutely nothing else.  These certainly don't pass as reality.

At the next major junction in painting/game history, our palettes increased, as did our fidelity of image, though our Art was still mostly flat as a board. The first known uses of shading in paint don't show up until Greek civilization, whereas in games objects occasionally get some darker pixels to depict depth.  Later 8-bit games portray a greater illusion of dimensionality but none are very convincing.

Figures became far more symbolic than the primitive silhouettes of prehistory.  Big simple shapes are used to denote important objects, like King Tut's hat vs Mario's mustache.  Even their poses and sideways-facing heads are remarkably similar.  Then there's the Ankh in Anubis' hand vs the cross on Link's shield; another interesting reflection of symbol usage.  Though, upon closer inspection of the Link sprite, I couldn't help but notice this:

They only had a few pixels to work with but still decided this element was important enough to be depicted in every variation of the sprite.  Interesting, right?  Which got me thinking, why does this look so familiar?  Well, let's just say I think we need to add a new page to the next edition of Hyrule Historia:

Anyway, we're getting off track...

Now we're starting to get a little more refined.  In both instances Artists have gotten more freedom to depict things as they'd like.  In painting, the invention of shading is wide-spread now and clearly shows; similarly more colors/bit-depth allows games to use shading more frequently as well.  Perspective still hasn't been invented yet, so it's uses are bizarre and limited in painting.  It's mostly faked in 16-bit games, leading to weirdness like this:

Trust me when I say, as a kid playing this game I certainly wasn't sitting there thinking, “man, I just can't enjoy a game with inaccurate perspective.”  Nobody noticed or cared; it's just an observation.

On a side note, later games of the 16-bit era, like Starfox, start using primitive 3D technology, which brings me to this:

Remarkably, both painting and video games figured out the full, correct usage of three-dimension space at about the same time in their respective histories.  Linear perspective was one of the great inventions of the Renaissance; it was documented thoroughly and became like a recipe that anyone could follow.  Meanwhile, 3D graphics came into their own and ushered in a still-continuing age of polygonal domination.  The early modeling of 3D lights also mimicked the move to the directionally accurate shading of the Renaissance.

By now video games had access to far more colors than the artists of the Renaissance did, but conversely, the painting master's had anatomically perfected the display of the human figure.  Our early 3D characters – sometimes accurate, often times deformed – are barely getting started.

So what was holding up painting in it's color progression?  The real-life limitations of pigment creation.  As you can see from this chart, the amount of colors available to artists was very slim.  They made the best of what they had access to, but they were certainly hampered by some of the items listed above: hard to reproduce pigments, awkward/toxic materials, cost to acquire, and so on.  Pigments were wild-cards.  One rumor is that Napoleon died from mercury poisoning due to the color his house was painted.  Hell, some versions of yellow were just cow piss; think of that next time you're at a gallery of masterpieces.  Plus, keep in mind that many of these old colors weren't permanent, so many paintings you see today are extremely faded versions of what they once were; a problem we don't have with video games, (though we have different problems when it comes to archiving).  Long story short: the next time you open a fancy color-wheel super-tool, just think how lucky you are.

Back on topic, we see a big jump forward this era in terms of lighting.  Both mediums began to correctly render light and shadow as we see it in reality.  The only other notable thing here is that the human figures in games have already improved substantially from the previous era.

Here we come to the final era of the Slow Ramp.  In both cases we've clearly reached a new level of mastery; a noticeable jump in fidelity that can't be ignored.  The shift is so great that our progress from here on out slows to a crawl.  While there are still advancements to be made, they're all minor additions.  This marks the beginning of a long plateau and also the start of The Divide.

THE DIVIDE

The yellow dots represent the eras we just discussed; the path that led us to The Divide.  This is where abstract art starts to be reborn.  While there were always artists pushing further towards realism, others were reacting against this trend, starting movements to suit own their interests, hence a divide.  The reasons for this divide are well documented in Art History, but interestingly, this divide also occurred in the game industry.  But before we get into that, take in this little slice of info:

Remember that these abstract works weren't the result of limitations, they were intentional choices by Artists to go away from realism.  And those who were leading the way in this divide were getting flak for it:

Sub-wallpaper was the description one critic used upon first seeing Impressionist painting.  People weren't much kinder to toon-shaded Link when he was first unveiled.  And although Van Gogh died broke, Windwaker went on to become a classic only a few years after it's creation.  In modern times we're far more open to abstraction whereas back in the days where realistic painting was king, intentional abstraction was the artistic equivalent to heresy.

So why invite this kind of resistance?  Why go against the flow?  Why the sudden divide in styles?  There were many reasons, some bigger than others, that led to this overall trend.  I've summed them up in this cumbersome, overstuffed image full of info (shame on me):

In painting, artists wanted to capture more of what they saw in day-to-day life, (mundane scenes, tricks of light), and less of the usual exorbitant, rich people in robes.  Around the same time scientists were starting to discover all the various hidden forces of the Universe, those that exist beyond the naked eye.  Similarly, spiritual texts talking of secret worlds and guarded knowledge starting gaining popularity, encouraging some artists to go beyond mere vision.  Finally, Western artists gained access to Art produced in other cultures, such as Africa and Japan, reminding them that there were other ways to approach the medium.

Meanwhile, in games, the sudden, staggering increase in costs to build competitive AAA games created a void where many companies/publishers perished and left behind tons of talented developers in their wake.  Around the same time, “simple,” looking games for the Wii, and also Minecraft, were seeing huge success.  People started realizing that absolute cutting-edge graphics weren't always needed.  This was apparent in the first games showing up on Facebook, Kongregate, and Newgrounds.  Finally, more and more mod/game making tools were becoming available, along with very useful video tutorials.  

In both mediums the divide kept growing at a slow but steady pace and may have stayed at that rate indefinitely if it weren't for a single grand event.  All it took was one invention to really get things going.  Modern venture capitalists would call it a, “game-changing, disruptive event.”  I call it The Fast Ramps.

THE FAST RAMPS

Look at that sweet angular action; art shooting towards realism AND abstraction at the same time.  We’re talking end-of-days crazy (am I the only one pumped?)  So what was the one invention that caused this?  What finally got us both extremes?  Well of course it's different for each medium, so let's look at painting first:

The effect the invention of the camera had on painting is well documented.  No longer bound by the requirement to capture reality, (“we'll never do as good a job as the camera, so why bother”), artists quickly took great strides towards abstraction, going beyond even the cave paintings of our ancestors.  Similarly, the camera brought us to the peaks of representation, giving us the final information we needed to masterfully simulate reality on a 2D canvas.  It led to this:

So what is the equivalent invention for video games?  My money is on this:

The App Store was the siren's call for thousands of out-of-work developers, entrepreneurs, and basement programmers.  Quick, cheap games made with simple, abstract shapes came out by the thousands, to the point where AAA games were the extreme minority.  We had fully returned to the simple looking games of yesteryear, and in remarkably short time.  For better or for worse, anybody with 3 cents in their pocket and half an idea could bust out an app and share it with the world.

Despite these advances, the smart phone had no effect on helping us achieve the photo-realistic reality we still wish to reach in games.  It brought us back to abstraction, but was no help in pushing graphics forward.  So what was the invention that brought games to full reality?

In truth, nothing has brought us there yet, and it's exciting to think that we still have new ground to uncover.  Ultimately, we're still shaping our history, and it's led to the scenario below:

While the lower ramp to abstraction is complete and already beginning to expand out into a wide spectrum of styles, (think indie games, where everything goes), the upper path is slowly plodding along, back in The Divide.  It's still making advances, still improving, but only inches at a time.  We still need the uber-invention that brings us to true graphical reality.  It could be a evolution of current tech, or something totally new.  Who knows?  Either way, it's something to look forward to, something that will change our world.  

In fact, we'll likely go beyond mere photo-realism – which is still bounded by a 2D plane – and reach Matrix-levels of three-dimensional reality, (notice the yellow lines I added to the chart above).  Sure, we have stuff like the Oculus, but not the graphics we need to make it sing.  And when we do it'll be sweet; we'll all know kung-fu and say “whoa.”  Until we reach that goal it'll remain a huge focus for our industry, but once we've climbed to the highest peaks we'll have to find a new direction.  Or many new directions.  We'll be able to pick and choose whatever artistic direction we want; that will be the day of true artistic freedom.  The days of flaming giraffes.

Until then, we keep advancing slowly, still reflecting the path of realistic painting.  With that in mind, this is where AAA games are today, in comparison to painting:

Like the captions say, we’re all about movement, action, physical simulation, particles, smoke, and so on, like the sweeping Romantic paintings of the mid-19th century.  We're making our worlds seem more alive by throwing in every trick we've learned.  And if we keep going by painting movements, next up should be realism, which despite what it's name implies, is really about depicting “real life scenes,” like people on a train, or working the fields, and not about representational reality.  I suppose Gone Home could be considered realism, but it's not really a trend, just a single work.  Which reminds me:

This overview is of major visual trends in both mediums.  Of course there will be individual works that don't fit the larger narrative, (Rez, Jet Set Radio, Ico), but this wasn't meant to be perfect.  You could also complain about my focus; why only Western Art?  The rest of the world was doing art too!  The answer is because that's what I know.  So feel free to expand on, contradict, or dismiss everything I've said.  It's all just an observation.  I was tired of hearing how games are like movies, so now you have some evidence to awkwardly compare games to paintings instead.  This is just me noticing how - coincidentally - my two favorite mediums have similar back-stories.

And it is a coincidence, totally a fluke; you think if Nolan Bushnell had access to the CRYENGINE back when he started Atari he would have said, “we're supposed to be on par with cave paintings, so lets pass.”  Hell no, he'd have used the latest, greatest tech in an instant.  But that's not how it happened.  We, like the great painters of old, have been fighting against technology for every inch of artistic freedom we can get.  And one day video games will finally reach The Spectrum (of Delight), where all types of Art will be created and accepted; where everything goes.

It will be glorious...and we're the ones who will make it happen!  Now find me a giraffe.

POST-SCRIPT or what, there's more?

In making my chart I noticed something: video games hadn't truly arrived at the furthest reaches of non-representation.  I'm talking Rothko's Black Paintings level of scrutiny where absolutely no symbolic mark exists on the canvas.  If you're not familiar, check it out:

Rothko was big on getting viewers to a spiritual place, not based on colors, symbols, or figures, but just purely by letting the canvases overtake the viewer.  He even said the ideal viewing distance was 18-inches away, so that the paintings were all that you see.  So I figured how hard could that be to replicate in a game?  This overwhelming purity of absolute non-representation.

With that in mind, I present to you The Black Game:

Yes, that's really a screenshot.  The game is absolutely black, with no HUD, sights, or sounds.  Once you start the game will run for 20 seconds then shut down.  If you press the SPACE button, the timer will reset giving you a random amount of time between half a second and twenty seconds.  If you keep tapping SPACE within half-second intervals, the game will never end, essentially letting you constantly choose how long you wish to remain in game.  Or you can try your luck, press SPACE occasionally, and see how long you make it, (of course, no feedback is given as to how much time you've just added).  When the game finally ends, you get to decide for yourself if you've won or lost.  It’s preferable to play with a VR headset so no light gets in, but otherwise, your face should be 6 inches from your screen.

Keep in mind that the black in the game is just black.  It doesn't represent a void, abyss, or anything like that.  That's the point, there is no representation here, but it is still, technically, a game.  Whether it moves you emotionally, spiritually, or whatever, is out of my hands.  I think it may actually stress people out, always worrying if they're about to be unceremoniously ejected.  But again, that's on you.

So how do you try this game?  Right now I haven't released it to the public yet as I’ve been doing a million things at once, including writing this.  Rest assured, the game will be free, as I can't see anyone paying for it.  I totally realize how pretentious this all sounds, but ultimately I just wanted to selfishly flesh out a weak-spot in my goofy Swordbill theory.  I don't expect any pats on the back for making The Black Game, it just needs to exist, so deal with it, yo!  If someone wants to release the same idea first, go for it. Maybe someone already has?  I’ll keep you posted when it’s ready.

James Beech worked on Weekday Warrior, DC Universe Online, Crysis 3, and ULTRAWORLD. He would much rather make a living painting, but that don't pay the bills, y'know.