Acting Your Way Over The Uncanny Valley

It is quite clear that given enough time, money and resources the uncanny valley can be conquered. Capture studios and developers are finding ever more ingenious ways of democratising the ability to bring cinematic detail and visual life to a digital face and body. All excellent and laudable stuff. Often excluded from the argument, in the fast evolving world of digital capture, the role of “suspension of disbelief”.

That phrase was first used by Coleridge in 1817. Suspension of disbelief is widely recognised as a  key component in any performance of a character on stage, in film, on radio, and in video games.

When you buy a ticket for a movie or a theatre show, on the whole, you are paying for suspension of disbelief – you can forget you’re watching enlarged 2d images on a screen, or an actor on a stage pretending to be the Prince of Denmark.  I believe it is the same facility that operates our credulity in other areas.  It has been shown that it is often harder to spot a lie from someone you already know and like, compared to someone you don’t know.  Its happened to all of us at one time or another.  Its true in other areas too: Scientists have to control for distortions to their research conclusions because of the effect of their need to believe their results will prove their hypothesis. Some call this “pathological science” The fact is, ultimately, we often believe what we want to believe rather than an objective truth.

The same is true of video games that involve character narrative.  Just as with film and theatre, there are scores of inconsistencies, illogical plots, and unfeasible accomplishments in every game, that go largely unremarked, because in the big picture, they are ignored.

Where suspension of disbelief  breaks down is when the medium is suddenly betrayed, and reality pours in.  An actor forgets his line on stage, a boom mike drifts into frame in a film, an avatar becomes incoporeal with a foot partially absorbed by a step.

The original observations by Masahiro Mori that would lead to the expression “Uncanny Valley” seem directly appropriate for digitally rendered characters. However its original remit was for the field of robotics, and studies often seek to standardise an objective reaction to near-human beings rather than take into consideration the effect on an observer who wanted to suspend their disbelief.  Look at the Uncanny Valley wikipedia entry and you’ll you’ll see a number of principles that have been proposed for avoiding the uncanny valley: “Appearance and behavior should match ability. In terms of performance, if a robot looks too appliance-like, people will expect little from it; if it looks too human, people will expect too much from it…”

So, there are various dangers when approaching the uncanny valley.  One is that the digital accuracy will fail, and the other is that the performance itself will ring false.  The observer then experiences “popping out” of that hard won immersive experience, which highlights the falsehood of what they are seeing.  The feelings of betrayal must also be a part of the “disgust” that Mori talks about.

In my work on Heavy Rain as Ethan Mars, I was fascinated by the capture process.  More particularly the challenge of performance “truth”.  I would perform an action, say going to a locker and taking a box from it.  There was no locker, but instead there was a wire frame hung on a hinged stand, in an empty capture studio. I would have to perform the action as if:
- there were rows of other lockers on either side
- that the contents were unknown to me
- that the box has some unexpected weight in it
- that I was nervous, worried that I was being watched, that I might be walking into a trap etc…

And quite a few other things.

Now this is the job of the actor – to suspend his own disbelief – and its something I’m quite used to doing. Eventually you develop a B.S. detector, both for your own work and the work of others.  My experience in TV and Film however, has shown that occasionally, the camera sees truth where a performance didn’t really provide it. In theatre, I haven’t noticed this duality. Which highlights something else quite important.

On Heavy Rain, if I was worried that something didn’t seem  right, I’d request to see the playback – not of the Standard DV record, but the point cloud of markers.  Because I very quickly realised that if something didn’t ring true, it was more obvious than on standard.  And this was the source data they’d be working from. I had no idea if all the added clothing, face, game design and the like would provide a smokescreen for any “unbelievable” clips they decided to use.

There’s a fairly logical reason for this. If we didn’t filter the information coming in our brains would approach a kind of overload.  Filtering what is, and isn’t, worth our attention is the best way of being able to interpret and react quickly.  However, this results in some curious ‘blind’ spots (Inattentional Blindness) It seems that the less information our senses have to filter, the greater the ability to detect falsehood.  This would be consistent with our lying-friend example, our feelings are also a filter, both what we desire and what we expect.

You don’t have close-ups in theatre, and you’re most likely to be at some distance from the actors; so again you’re often judging on a whole-body view, and therefore I feel that it has more in common with this part of performance capture than film does. Your suspension of disbelief is reliant in part on a physical truth.

Which brings me back to the uncanny valley.  I believe that now we have better technology that can bring highly detailed and subtle expressions and behaviours into the digital world, it is the quality of these performances that will provide the glue that keeps the observer/player immersed in the experience. As film and theatre directors know, technical proficiency alone cannot give the audience an experience that goes much further than a great fairground ride.

A dramatic narrative well told, then, will be the difference between a piece judged on the limitations of its media, and a piece judged on its merits as an emotional experience.  To some degree we have already seen this in Heavy Rain, where its technical faults are more forgiven by those who have an emotional attachment to the narrative, meaning the performances, plot, and ‘truth’ were more likely to be critically examined.

I was at an industry conference recently, and one of the panelists insisted that his best advice was to never try and mix the rules of the media you’re working in.  TV?  TV rules.  Film? Film rules?  Video Games?  Well this is where I differ.  In this brave new world of convergence, it is premature to believe that the rules for Video games have been ‘set’.  I would say its wiser to establish what Video Games have in common with other media, before excluding anything from the argument. This holds true for the Uncanny Valley.

[This blog post and others like it can be found at the Motives In Movement website.]