Gaps of the Imagination

Previously posted on the Frictional Games blog.

At the very core, games are driven by systems. We have extremely powerful machines that can quite accurately simulate various forms of physics, economics, group dynamics, intelligence and so on. This means that at our fingertips are the tools to create entire worlds. It is therefore very tempting to use these in order to solve every single problem. But this is not always the best approach. Sometimes it's best to just leave gaps, and let the player's imagination handle the rest.

There are five core reasons why this is so:

All of this works because of how our brains is constructed. We are all equipped with highly capable pattern-matching machines. Our brains, and therefore we ourselves, want the world to make sense and will always try and fit things into a coherent and convincing narrative. The simplest example of this at work is motion.  Just consider this animation:

(image source: Pilar Newton at tutplus.com)

This is really just a series of circles that we see in rapid succession. Despite the fact that it's quite easy to notice the individual frames, we still see it as a ball jumping up and down. We fill in the gaps between the frames and see it all as an animation. Notice that this doesn't really take any conscious effort - it comes naturally. In fact, it is quite hard (possibly impossible) to not see this motion. When done right, gaps of the imagination never really feel like such - they simply get incorporated into the overall experience.

Here is another classic example:

There is no white triangle pointing downwards, but the other shapes suggest that there should be, and hence we see one. Again, this is something that we carry out automatically and while we can rationally understand that there really is no triangle, we cannot mentally unsee it.

Closing gaps like this is not just a fun side effect. It is a crucial way in which our brains function. Taking decisions on incomplete information is an essential skill in the function of our day to day lives. In fact, most of the information we receive is incomplete - it's quite rare that we are directly exposed to situations where we have all possible information to hand. So it's not so strange that closing gaps should come so naturally to us, we literally wouldn't be able to survive otherwise.

As I have said before, any decisions we make are based around a mental model. In our heads is a mental simulation of how we think the world works, and before we do anything we first run through the action in our model to see if it would have the effect we intend. However, it is not possible to get complete information about our surroundings. So in order to get a working map of the terrain we need to fill some of the information. That is what the gaps in imagination are all about - our process of crafting a working mental model.

Given that this is a foundational part of how people work, it is essential that we always have it in mind when designing games. After all, the end result of a game is really to create a specific mental model in the player. This can be done through direct system access, sensory input or, what this post is all about, through suggestions of something unseen.  I will now go over the most common, and most important, ways in which gaps can be used.

---

Sensory Gap

This is the most common one and also easiest to understand. When we are presented with sensory information that is somehow incomplete, we try and fit it into a working context. There are many different ways in which to do this. It can be quick, disjointed shots of something, events seen through the eyes of a monster, a character reacting to a smell. or a sound that is so loud it shatters glass. Normally you build this sort of gap up by presenting a negative space where one or more object/events/reactions/etc. all related to something unseen. 

We tend to use this sort of gap more than we realise, simply because it is such an ingrained way of working with various artforms. But that just means you have to be extra aware of how it works. For instance, it's good to know that we are extremely prone to seeing familiar shapes, such as faces, in just about anything. 

It's actually quite hard to not see anything in an image. We have evolved in an environment where there was nothing like pure noise, everything we saw was part of the environment somehow. Therefore, when we see random patterns the brain goes into overdrive to find a connection. Of course, it is not just the direct information received that we use to do this, but anything related is accessed as well. Because of this, you can often get away with very little if the scene is just set up in the right manner. Learning to do this is incredibly important.

Informational Gap

This is very similar to the sensory gap, but instead of sensory data it deals with information in the abstract. Jeff Vogel recently wrote a blog post on how games often have too many words [1], and properly using gaps is the best remedy for that. It is often easy to underestimate just how much information a player is prepared to fill out for themselves. You might think that a player really needs to know details of everyday routines in order to fully understand how a certain society functions. But often it is just as good to simply have some brief glimpses (visually or through text) of what they are doing along with something that sets the tone (music, animation, etc). 

This sort of advice might seem a bit obvious, but it is worth keeping in mind how good we humans are at filling out these sort of gaps in information, especially if it comes in form of gossip. Gossip is something that has probably been part of our lives since before we evolved into modern humans. We are social animals and gossip serves as a tool to keep track of the group we are living in. This means our brains are extra-sensitive to this sort of information, similar to seeing shapes and motion, and can easily assess even the smallest hints.

Spatial Gap

Games are often very spatial and requires the player to navigate various places. Because of this it is often very tempting to accurately represent every single area to give a proper sense of place. However, for various reasons this might not be feasible or required. It is then important to note that spaces can also be created from gaps, just as much as sensory information and lore. Spatial gaps are a bit trickier though as we are now messing with the actual play area which means you you have to be extra careful.

The sloppy way to implement is to simply have a lot of locked doors that the player cannot access. But this can be bad for pacing and, worse, it can lead to a weak mental model. The brain is usually quite lazy and will try to optimize. If the pattern becomes 'locked door equals the room is irrelevant', it will be noticed and incorporated into the model. After all, dismissing these spaces is much easier than assuming they are there.

Instead it is better to vary the ways in which the rooms are blocked off, and to build up a negative space hinting that it is there, such as glimpses into it from a small window. It is especially effective to make the player believe that room might contain something of value. That way it is much more likely it will be part of the mental model. Remember, what matters is not what is actually there, but what the player perceives to be there.

Causal Gap

This next time is very similar to the animation example from above, but happens over longer time scales. As I said before, closing gaps is all about creating a working mental model. Where the previous gaps where are all about what we see, causal gaps is more about why we see it. Just like we humans are quick to see familiar objects in random patterns, so are we quick to correlate two events with one another. A black cat runs over the street and then you get a headache. We humans often correlate the most strange occurrences and it is quite hard not to. Many of our psychological fallacies are really based around this fact, and it takes effort to learn not to do this. Why is this so? Because for our ancestors it was okay to have a few weird beliefs if it meant we could pick up on dangerous situations and survive better. If you are a skeptic about the cat causing the headache, you might also be skeptical towards the signals of an impending avalanche. 

This is obviously the sort of brain glitch that we can make use of in art. Film is a medium that uses it all the time, in editing. See a gun fire, and then a person being hit, and it is obvious what has happened. Correlating these two events into a single event comes naturally to us. But because games are so driven by the player understanding the flow of input and output, or how the player's actions correlate to events in the game world, it gets a bit harder to deal with gaps.

The simplest way to do it is with events taking place. For instance, say the player suddenly notices that a door is blocked, and hears the sound of feet running away. They will now assume that whoever is running away is also the one who blocked the door. 

A little harder is for us to provoke imagined causal connections from actual interaction. What I mean by this is that you actually leave out certain information when actions take place, and ask the player to fill in the middle. For instance, if I click on an object in a game and it simply pops into my inventory we would like the player to mentally simulate this as "I picked up the item" and not "the item got magically teleported into my inventory". The three key elements for achieving this are consistency, negative space and optimization-avoidance.

Consistency means that we need it to happen in a way that makes it possible to distil it into a couple of universal rules. For instance, is the distance you can pick up items at consistent with the player using their hands? Negative space means that we use other events to reinforce the fantasy. These can be sounds effects when you pick up, feedback messages when you are too far away, and cutscenes where we see this happening for real. Finally, optimization-avoidance means that our intended mental model, that of the player actually picking these objects up, must be the "simplest" [2] available explanation. If there are too many edge cases, weird behaviors or simply not enough supportive negative space the "magical teleportation"-theory will win.

Anticipatory Gap

This one is quite similar to causal, but worth having in its own category. These are basically events that might happen to the player in the future and that they will take into account when planning. As explained in an earlier blog, planning is a core reason why gameplay is engaging, and thus it is important to shape what sort of consequences the player can conceive of taking place. When the player plays a game, they will generate their mental models not just based on what they are playing, but also what they know from before. This is gives us an opportunity to make the player think certain things are possible, without them never having witnessed them.

Suppose that the player hears footsteps from afar. These might just be coming from a couple of scripted sound effects, but the player doesn't know that. Given the right context the player will conjure up a monster that is making these sounds and assume that it might attack them. Now the player will start making plans based on a few sound effects and project a lot more onto the game than what is actually there. This is something that we saw a lot in Amnesia. The player could create long, and engaging, gameplay situations for themselves only because of a few sound effects. 

Of course, this sort of trick can't last forever. As I have mentioned several times, the brain likes to optimize and once the pattern gets to clear the illusion will go away. So it is important to constantly update the negative space, to not make the events so predictable and to setup situations in ways that feel exciting to partake in. Even more important is to make sure that this trickery is not the only gameplay there is. If the player has a baseline of actual planning  and execution going on, this sort of illusory anticipation can be sustained for quite a long time and add a lot to the experience.

Agency Gap

If you are in the jungle and the high grass suddenly moves. Do you:

In our ancient past, the people who thought like number 2 were much less likely to survive. Sure, they escaped the embarrassment of being afraid of grass from time to time. But they also got eaten a lot more by tigers. Thinking of events as being caused by something with agency (e.g. an animal) is powerful concept for survival. It is also something that leads to all sort of weird beliefs like tree spirits and demonic possession. For many events it comes naturally to think of them as caused by intentional beings. And once again, this is great stuff to use in games.

This video that is a great example of the whole thing in action. Just note how you directly project agency on the shapes.

The most common use of this is in enemy AI. Most of the time, AI that feels smart is because the player thinks it is so, not because the underlying systems are complex [3]. F.E.A.R. is a great example of this. While the AI does have some clever systems at play, it derives most of its impressiveness from being good at giving the player feedback. For instance, by saying "cover me" while a grenade is thrown, it gives the impression that much more is happening than what actually occurs. The player projects a set of thoughts running through the soldier's mind and incorporates that into their mental model. But in reality it is just very simple code that is executed.

This is an area where the brain works a lot to our advantage. We are ill-equipped to mentally simulate things like state machines, but it comes naturally to think in terms of people. This means that we can get a lot of content to the player only through suggestion. The player's brain is really apt at simulating people and can do so much better than any existing computer system. So the more of that power we can use, the better.

All of this get harder when we get away from combat though. While players want to think of other characters in games as entities with rich mental lives, there is a limit to how far they are willing to go. If animations start to look weird, if responses come out as canned or if interaction possibilities are too limited the brain gets lazy and optimization kicks in. Characters go from intentional beings to simply being objects.

A big problem of interactive storytelling is how to keep this from happening. Just as I discussed with causal gaps, the key elements in achieving this are consistency, negative space and optimization-avoidance. How exactly to achieve this with characters is still too unclear and complex to cover this time around - I will go over some possible directions in a future blog post.

Volition gaps

Finally we have arrived at the last gap type, which is similar to causal gaps. Previously we have mostly talked about things happening when the the player is the one causing it, but volition gaps contradict that. These are events that the player thinks they cause when in fact they don't.

This is something that I think currently is quite unexplored and has a lot of potential. It is quite hard to construct control interfaces that allows the player to take all the possible actions. Therefore it would be nice if we could have actions that the game does automatically but that the player believes they used their volition to cause. This is what this gap is all about.

Just like we construct a mental model of what is happening on the outside world we are also constructing a narrative of what we are doing ourselves. While we are not conscious about it in everyday life, a lot of the time we have reasons and beliefs because we do certain things, instead of the other way around.

For instance, take choice blindness. In an experiment the subject had to pick which face they thought were the most beautiful, and then when handed back a different picture, most subjects continued to explain why they had chosen it. So a lot of the time, we make up reasons based on the actions we find ourselves doing. And as always, whenever there is a brain deficiency, we should try and exploit it for art.

The best example of this effect in use is in Assassin's Creed. When you run across the roofs in a city, the game will automatically make the character jump. Despite that, it always feels like you are the one who is willing those jumps to happen. This works as a great way of streamlining the controls and making the experience more fluent without taking away a sense of agency in the player. Another example of is in games like Uncharted where the character will interact with the world in ways that make sense and enhance the feeling of being there.

I believe that these sort of gaps can be taken further, though. An interesting example of this is The Path, where the protagonist will carry out actions of their own choice when you leave them alone. It is by no means perfect, and doesn't really provide an illusion of volition. But I think it shows the potential of this and I think causes the player to feel intimately involved with various events if used correctly. Currently, though, it is too unexplored to say for sure what the possibilities really are. I will explore some more thoughts on this in a later blog post.

---

That should summarize the basics for the various gap types. I am sure there are even more than these, but this selection is what I found most useful for games. In upcoming blogs I will dig deeper into some aspects of this and give more examples on how it affects gameplay.

Until then I recommend reading Ian Thomas's excellent article on the subject where he discusses gaps for LARPs, SOMA, and much more.

Footnotes:

1) The follow-up post that looks at the game Pillars of Eternity specifically is also well worth reading: http://jeff-vogel.blogspot.se/2017/06/games-have-too-many-words-case-study.html

2) This is simplest in terms of effort used by the brain, not in a strict theoretical sense. For instance "a witch did it" is a simple explanation for a human brain, but in reality is quite complex because it assumes a lot of attributes for the witch. But we as humans are great at just ignoring this, making the silly explanation seem simple.

3) This video shows it off nicely: https://www.youtube.com/watch?v=9bbhJi0NBkk