This post is edited from a version on the writer's development blog, where he talks about his work developing installation-based games and playable performances.

In 2015 I developed a game called Codex Bash. It's a codebreaking game for a group of up to four players, where they have to input coded sequences by pressing coloured buttons spread around a room. Paper sheets and props are spread all over the play area, and must be used to decode different puzzles. It's very much like an escape room that fits into a rucksack and can be played in short frantic bursts.

It's a game I'm very proud of, and it won an award at IndieCade that same year, which I'm also very proud of.

In the early versions of Codex Bash, I really struggled to make players notice the props. Realising that I should spread them all over the room was a "eureka" moment that transformed the game.

I'm writing this article to talk about perception, how we perceive what's in the world around is, and in particular what that means for games. I want to talk about how we as designers can overlook the limits of player perception, and suggest that understanding this gives us a chance to create surprises.

To do this, I shall start by explaining how I got to that "eureka" moment, and what I noticed along the way.

Fixing the Puzzle

Before this point, all the paper clues were in the one booklet, pictured below. Hardly anyone could solve these puzzles without being told, by me, to open the book.

I changed the user interface over and over. At one the point where the screen would show a picture of the schematics booklet, and the booklet itself would be in clear view right next to the screen. Yet players would still stare at the screen for ages trying to make sense of it. They would look directly at the booklet and stare back at the screen again. Back to the booklet. Back to the screen. They never touched it.

What was failing to click with them?Did they think the solution had to be entirely visible on the screen? Did they think the booklet was simply left behind by the tech crew and not meant for them?

Whatever the reason, the booklet simply didn’t exist in their mental model of the game. So interacting with the physical object never even crossed their mind.

They looked at it as if it didn't even exist.
 

The joy of Codex Bash is all the little "eureka" moments in each bite-sized puzzle. At the same time, if you ask players to make two mental leaps without feedback they feel flummoxed - especially when under pressure. Reward each individual leap and they feel smart. I'd not realised that realising physical objects were actually part of the game itself is already a mental leap.

Before they can make this particular leap, players need to be aware that there even are physical objects of unknown purpose around them.

I tore the pages out of the booklet and spread them around the room: next to the buttons so it was obvious that players would see them by accident. It worked. For the next event I laminated them, so they were clearly not flyers or pieces of litter. It worked even better. Incidentally, this also made them beer-proof.

So here's where I got fixated on the difference between what we see and what we perceive. Is not perceiving an object in the game world congruent to it not existing at all? Is our awareness of an object being in a game dependent on whether or not we think it serves a purpose?

Object Permanence and the Magic Circle

Object permanence refers to the idea that even when we cannot see something we are aware of its continued existence in our world. It is a learned concept. It is a skill we acquire as infants.

When you play "peekaboo" with a baby it is surprised at the shocking reveal that, yes, you were hiding behind your hands all along. When a baby cannot see you you do not exist in its reality. As far as it is concerned you have disappeared entirely. It has not learned object permanence.

Indeed, scientists often talk about how much of our perception is based on our minds filling in the gaps. It's why optical illusions work. So if what we perceive in the real world is based on what we assume to be there, is what we see as part of the game world based on what we assume to be part of the game?

In game design we often talk about the magic circle. When we play a game together we suspend our notions of normal behaviour. As players we silently agree to a new reality. Within the magic circle it’s okay to be silly, or to make physical contact, or to lie, depending on the premise of the game.

Within the magic circle, we have a concept of what exists in realities inside and outside the game. Hunger, passers-by and trip hazards exist outside the game but continue to be relevant to us. No matter how engrossing the game, these objects remain permanent in our mind because they are important.

In a typical screen-based game - Super Mario Bros, for example - question mark boxes and goombas belong in the game world. Similarly our thumbs and the buttons on our controller are part of the game world, as they are essential to the game functioning. You would be well aware if your sweaty thumbs slipped off the buttons. My belief is that our mental model of the game world is based on what has function within the game.

In the X-Men game on the Sega Genesis, one level required players to use the “reset” button on the console to move to the next level. It's an idea that sounds surprising and fascinating. I admire its creativity. But for so many players this was game-breakingly frustrating. The reset button is used for the mechanical functioning of the console - turning it on and off - and so in our mental model of the game it does not exist.

Why would you even think to press it? Its function is to switch off electronics. It is part of the real world.

All players are aware that the reset button exists. We can probably see it right in front of us. But it has no perceived function within the game. So it simply does not exist in our mental model.

When players were unaware that the booklet was part of Codex Bash they usually acted like it didn’t even exist. I watched this happen so many times, and it fascinated me. Even when they looked directly at it opening the booklet, even touching it, did not cross their minds.

And it was perfectly logical not to cross their minds. As far as they were concerned it didn’t exist. Not in Codex Bash world. Stepping into the magic circle, it appeared to serve no purpose. It did not belong to the set of things involved with solving a video game.

The booklet existed in a reality: a reality that they could see but not in a reality they actually cared about.

Virtual Realities

I was in Abu Dhabi last September for an A MAZE popup, and was speaking to its director, Thorsten S. Wiedemann, about VR. In January 2016 Thorsten spent 48 hours in an HTC Vive, and his experience was filmed for a documentary.

We chatted about the future of VR and his experience, and I asked him what his favourite VR experience was. He told me it was the chat rooms. Despite being filled with identikit human models sliding around a bland space unrealistically, he was connecting with other human beings sharing his experience. That social contact made it more real.

I believe that in more standard games, more akin to quick-fix experiences, the illusion of reality was fleeting. Social contact gave that reality a reason to matter.

My own experience with VR is limited, and I'm sure there's people reading this who are more experienced creating work for it and observing players in it. Nevertheless I’ve personally found the sensory deprivation - brought on by the headset and earphones - to be as powerful as any other aspect of the techlology.

I can always tell I have a screen very close to my face, regardless of how high-resolution that screen may be. I know that the polygonal ground is not real, that 2D particles are flying in front of my face, and that the virtual arms in front of me only vaguely line up with my own. None of this is important.

When I have no engagement with any other stimulus, the virtual world in front of my eyes may as well be the only world there is.

This was clearly the case when playing Lucid Trips at the A MAZE festival back in April 2016. I knew there was a crowd of people watching me, but I did not care how stupid I looked, jumping up and down and flailing my arms wildly.

My attention was entirely on one goal: I wanted to fly.

I was fully aware there was a whole physical world going on around me, where I looked horribly embarassing. But my emotional desire to feel the sensation of flight was way more exciting. Additionally, to perceive the physical world outside, one that I could neither see nor hear, required extra effort. The real world lived on in object permanence, but I simply did not care about it as much. Flying was my purpose. The virtual world could create the sensation of flight.

It was purpose that people could not see in the Codex Bash booklet - how is a paper booklet part of a video game? It was purpose that people could not see in the X-Men reset button - how is a button used to switch the console off used to play games on it?

It was purpose that made me - rather than lose awareness of the real world around me - simply care about the virtual world much much more.

Purpose Makes the Reality

I propose that our perception of the world around us is not based on what’s actually there, but instead on what matters to us - be it on an emotional or practical level. It’s important to understand this when designing all user interfaces. Not just in VR games. Not just in installations.

It is important in any project where we want to create immersion, whether we want to immerse users in a world or immerse them in a task.

To convince me I am inside another world I need not be convinced that it is real. I need to be convinced that I should care.

Perhaps I care because I get to prove something to myself and the people around me. Perhaps I care because the story and setting help me reflect upon similar situations my own life. Perhaps I care because I am isolated from the world of worries outside the game.

To find out why they will care we need to be aware of what’s going on in our player’s mind. What do they know? What do they want? What will they infer based on prior experience?

Once we can answer these questions we can build a picture of what our reality looks like in our players’ minds, and we can reach them on a deeper level.