It's All in Your Mind: Visual Psychology and Perception in Game Design

Author: by Hayden Duvall

Making a game is sometimes like being in a bad relationship. In the beginning, everything ticks along nicely and your partner seems happy as you try and provide for their needs. You both know what you want, and all seems well with the world. After a while however, your partner begins insisting that you spend more time with them, you never seem to be able to do enough, and they become more demanding. Your life soon becomes dominated by a never-ending list of things they want. Fail to give them the care and attention that they dictate, and they stubbornly refuse to do a single thing that you ask of them. Eventually you give in to their requests, secretly plotting to end the relationship as soon as you can, moving on to something better that will allow you the space and freedom you need to truly express yourself. When the split finally comes, it is a relief, but looking back, seeing all the things you achieved together, you realize that through pain there is reward. Next time, however, you won't be pushed around quite so easily.

You may think that this article will address that most insidious of evils: Feature Creep (which to me always sounds like one of Spiderman's less successful foes). This however, was not the purpose of my "a game is like a relationship" analogy. My intended point was that a game (like a relationship) is a complex, dynamic thing that requires the participants to draw on all areas of understanding to make it successful. Similarly, the more involving and well constructed the game world is, the more a player will be drawn in and rewarded.

It is from this point of view that I propose that psychology has a part to play in game development. OK, I can hear you groaning. Psychology: the realm of ink blots and Freudian slips. But stifle that yawn and bear with me. Psychology aims to explain how the mind works, and how this leads you to act, think and experience everything from falling in love to dressing as your mother and attacking women in showers with a carving knife. Whether your game is trying to be a perfect simulation of flying an F-16 or an epic adventure set in a unused carpet factory, understanding psychology can improve your game's design and execution.

Psychological techniques have been effectively used by video games for years, simply because we all live in the same world and decode our surroundings using basically the same physical and mental machinery. Our life's experience brings us into contact with a similar range of emotions, and it is this framework that we draw upon when we create something, whether it be a book, a song or a game. What I suggest is that we put names to these psychological aspects.

So, where to begin? Let's see how the player interprets what they observe and hear.

You Think That's Air You're Breathing?

First of all, a brief overview:

Theories of perception generally draw upon the same basic idea. Gregory's definition reads:

or more succinctly put:

The model is perhaps an obvious one, and as far as creating a game goes, we only need to concern ourselves with visual and aural input (with the tiny exception of force-feedback controllers). This restriction puts extra pressure on the visual and audio aspects of our game, and removes the need for us to create a convincing "smellscape".

It is worth pointing out that impairments to a player's senses can impact a game. At the most basic level, deafness rules out most dialog-heavy games that don't have subtitles, as well as making games in which audio cues form an important part of the gameplay all but impossible. Colorblindness is also a rarely considered, yet significant condition, which can affect a player's enjoyment. Red and green color blindness is the most common form, and while it only occurs in 0.4 percent of the female population, it is estimated to affect eight percent of males, and as gamers are still overwhelmingly male, this figure is not inconsequential. As this form of color blindness restricts the sufferers' ability to differentiate between red and green (hence the name) any vital information in a game that requires color matching (puzzles) or easy identification of colored objects (the baddies are wearing red, the goodies green, for example) could be enough to render the game unplayable.

Seeing is Believing

The first and most obvious area of perception is vision, so this article concentrates on this particular type of sensory input. If I had been writing fifteen years ago, this section would only be about two paragraphs long, as the limitations on game visuals would have left me with little to say. As it is, today's games are so visually complex that what we see on the screen is beginning to rival the intricacies of the real world. Yes, it is true that a game world has significant technical limitations as to the content of a scene, but when we consider that the range of things that a game can present us with visually extends way beyond that which we will ever see in the real world, the balance begins to be redressed slightly.

Consider for a moment, the following images:

While these kind of illusions may not in themselves be of any practical use in game design, they illustrate the process by which our brain translates what it sees (stimulation of retinal cells) into what it believes this information represents, and what this means. Clearly as these images show, what we think we see, is in fact not necessarily what is actually there. Fortunately for those of us making games, our brain can be tricked into thinking it sees a whole range of things that are, in reality, no more than a collection of glowing dots of color on a flat screen.

There are quite a selection of rules that influence how we interpret what we see. Some, like binocular disparity (the fact that our eyes are slightly separated and so receive slightly different images), have no value in the average game world (put those 3D glasses down). But certain others are of more interest.

Similarity: objects that appear the same or similar are grouped together in our mind. This can often be reinforced in a game when animation cycles of several similar objects run in synch: trees wave in the wind in unison, torches all burn at precisely the same rate. Sometimes grouping may be what we want, but more often than not, this similarity just serves to highlight the fact that some elements of a game are repeated.

Relative brightness: as things get further away from us, they tend to fade and take on a bluish hue. This phenomenon is used as a depth cue, and is of course great news for fans of fogging. The trick though, is in the subtlety and distance of the fade-out. If the fogging distance is too close, and the drop-off too severe, instead of an acceptably natural haze that reinforces the expansiveness of a landscape, the player will begin to feel that they have wandered onto the set of a 1930's Sherlock Holmes movie.

Scale: perhaps the most important aspect of scale is that it is relative. Perceptual consistencies, as psychology calls them, dictate that elephants are large and gerbils are small. No matter what the viewing conditions, our past experience provides us with this sort of information, which is used by our brain to decide that the elephant on a distant hill is smaller than the gerbil in our hand (Helmholtz, 1909). In a video game of course, elephants can be two feet tall and gerbils can weigh four hundred pounds. The rules of consistency are of little consequence. In addition, everything we see is effectively in screen-space, and will actually be observed in a physical sense, as being only inches high. In view of this, context within the game environment, has a large part to play.

In the example above, the central gray squares within each of the four larger squares are exactly the same. The change of brightness across the four surrounding squares however, tricks us into seeing a corresponding (but inverted) shift in brightness across the inner squares. The effect of context dependency is also valid within a game setting, when dealing with scale.

Size, relative to the player's character (or vehicle, etc.) can be used as a starting point for scale comparisons. What appears to be a human-sized character will be interpreted as such, if the surroundings reinforce this with appropriately sized trees, cars etc. If however, the character has to climb a can of beans 10 times his size, or run across a massive piano keyboard in a Tom and Jerry kind of style, the character may take up an identical amount of screen space, but will be judged to be tiny. In the same way that geometry sizing can convey scale information, textures can also have a part to play, with the scale at which a texture is applied to a surface, having a contributing (if sometimes subconscious) effect on a players sense of scale.

Speed: moving things rapidly around the screen may seem like the obvious solution in this case, but speed isn't just about being quick (not when it's on a screen at least). Let's pause for a second, and consider what most of us see as Lee Major's career high: the Bionic Man. Apart from an impressive line in polyester sportswear, one of the bionic man's most important assets was his speed. TV budgets being what they were in the Seventies, as well as the limited range of special effects on offer, the bionic man's production team had to come up with an acceptable way of making him fast. So what did they do? They slowed him down. On the one hand, this perhaps explains how the logic of this decade brought extreme chest hair into fashion and made Elton John famous, but it also makes sense. TV and film had already established the concept of slow motion, and it was of course predominantly used to show something happening too fast to be appreciated. Drawing on this convention, the bionic man gave us something that was visually slow, but translated by our brain into extreme speed. So what does this imply for creating the impression of speed in a game?

Essentially, we can use things that we associate with speed (like slow motion) as indicators that something is fast. Motion blur is perhaps the most widely used of these. Once only available as a pre-rendered effect, we now see it executed in real time. Apart from the fact that motion blur actually produces a moving image that the brain accepts as more convincing, we have now become familiar with the convention that blurring equals speed (in this respect I am talking about exaggerated, visible blurring). Roadrunner cartoons are particularly fine examples of this, building on the traditional static cartoon methodology for conveying speed. A more recent example in film would be The Matrix, where extreme, bullet-dodging speed is expressed with an adapted form of motion blur. Other visual cues to speed would include things like clouds of dust and debris in an object's wake, or the intensity of flame from a spaceship's rockets increasing (the afterburner effect). Indicators like these will help steer the player's brain towards association with speed, making the effect both easier to achieve and more convincing.

Once we are able to perceive the game visually, and identify its constituent elements, the next trick is making sense of it all. At this stage, there is a whole world of pain, ready to be explored when looking for a model that explains how we do this (see: Hochberg 1978, Koffka 1935 and Marr 1982), but perhaps of more interest to game design, is how we can help the process along.

Linearity in a modern game seems to have become as unpopular as a mime artist in a …well, as unpopular as a mime artist. The days of simply routed games, where all you needed to think about was collecting giant golden keys and making your way towards a fight with the big red dragon are long gone. Free roaming, unrestricted exploration is the norm nowadays, with complex environments that allow the player to wander for hours, marvel at the scenery, and cavort across hilltops with a Julie Andrews-like abandon. But freedom comes at a price.

Dropping a player into a strange new environment can be confusing. The amount of information presented can mean that a player becomes unsure about which direction to take and unless some kind of help is available. The appeal of free exploration can be crushed under the tedium of random wandering. Solving this problem relies on good design and well-executed construction, but there is one additional area that plays a part: memory.

At the Copa… Copacabana…

Remembering where we've been, what we're doing and where we are going can (for some of us) be a difficult feat in our everyday lives. I for one often find myself staring blankly into the fridge, having no recollection of even entering the kitchen, let alone what I am looking for. Add a story line, several characters, a world full of caves and a sacred quest of some description, and suddenly, keeping track of what's going on becomes quite an issue. There are a couple of memory related problems that often crop up in games, which can in my opinion, reduce a player's enjoyment considerably:

First, there's the common fault of information-induced panic (or IIP). We've all played games that from the very instant they start, begin to heap upon the player what seems to be vital information. Whether it's in the great hall of Castle Colon, or in the transporter room of the Starship Artichoke, there are always characters ready to give us what seems like thirty minutes of introductory background story and a comprehensive rundown of our mission. Unfortunately, you might mix up the fact that our sworn enemy Thoth'l, son of Thathel, has captured the Princess Thath-oth and is taking her to his lair just outside the town of Thimbar, with the fact that our trusted guide Kaboth'l has located the once powerful Sorcerer Thim-Athlothal, and has recommended that you travel to Hack-Matheth to find him. What we end up with is the vague impression that we need to find someone who is going to help us rescue someone from somewhere, and that we may need to do some fighting on the way.

I have been facetious with all of the "th" sounds, but this scenario is one that often pops up, but firing out character and location names which obviously mean nothing to the player, can strain layers as they trying to take it all in. True enough, many games that do this do not expect you to remember everything, and much of the information is meant as nothing more than supportive detail to the central plot. The problem is that players don't know this.

The second memory-related problem is exploration anxiety. As mentioned previously, the trend away from linearity towards free exploration exacts its own peculiar price on the player. This anxiety comes in two basic forms, the first of which, as mentioned before, has the player wandering aimlessly around a world, searching for something specific (or unspecified- in the worst cases), unable to remember whether or not they have been along this particular path before. Unproductive rambling can drive any player towards the escape button, but coupled with the common game-world problem that trees, corridors, paths and caves can often look the same (regularly using the same geometry), exploration can sometimes be its own worst enemy. The other form of Exploration Anxiety that most of us will be familiar with, is the nagging feeling that sometimes appears during a game, niggling at the back of your mind saying "I bet you missed one of the rooms back there", and "I bet there was a huge magic sword in it". Progressing from level to level is generally the driving force behind the gameplay, but the thought of having missed something vital, or of significant value along the way, can sometimes be detrimental.

So, what has psychology got to offer to help game designers cope with memory issues and their associated problems? I would suggest a look at the following three rules.

The Magic Number

Funnily enough, this number is seven (Miller 1967). Or more precisely: seven plus or minus two. In psychology, things are only talked of in terms of theory, the Law of Penile Envy for example doesn't exist in the same way as the Law of Gravity. Even though certain elements of thought and behavior seem to be well established, psychologists are happy to regard them as theoretical and not proven. The concept of a magic number is however, one of the most robust of these phenomenon, demonstrable across many cultures and age ranges.

Try it out for yourself:

All things being equal, your recall should have fallen between 5 and 9 letters: the magic number. There are a range of interesting modifiers to this phenomenon that allow recall to extend beyond the magic number. Chunking for example, suggests that we place the information into lumps (or chunks even) and thus the overall amount that we remember, is increased. As an example, it would be relatively simple to remember the sentence: aliens ate my best-friend's underwear, which in actual fact gives us a total letter recall of 30. This chunking process allows letters to be made into words. The same trick is used with phone numbers -- businesses want us to remember 0800 60 40 2200, for example.

This principle is probably most applicable in a game, to puzzles. Using too many elements at any one time will most often result in the player loosing track of what they're doing, or will force them to begin writing things down.

Primacy and Regency

This is best illustrated by the following exercise:

Typically, the words at the beginning of the list and those at the end are most easily recalled (Murdock, 1962).

Primacy therefore, refers to the effect that makes the earliest words easier to remember, and regency the enhanced recall for words at the end of the list.

How does this help game design? It can be established that once the amount of information presented spills over the normal limits of short-term memory (seven items, plus or minus two), the middle ground becomes fuzzy. In terms of cut-scenes and long-winded intros: if they serve to build atmosphere and establish character, fair enough, but place any important information in the beginning or the end.

Repetition

However unreliable my memory is, the two things for which my recall is perfect are the words to Fiddler on the Roof, and the monument to modern song-writing that is Barry Manilow's Copacabana. This is not due to either an obsession with musicals from the '70s nor too many nights spent at karaoke, but rather my parents' unfortunate taste in music as I was growing up. Whether we like it or not, repetition is a reliable method for reinforcing memory, and is an effect that can be used in games.

On the most basic of levels, repetition of important story elements helps the player remember details that could at first be difficult to retain. In terms of familiarizing a player with a control system, or having them learn certain procedures within a game, repetition is both a way to help the player remember what to do, and provides the opportunity to practice these actions. As long as the repetition is skillfully paced and well integrated into the overall flow of the game, it should do its job without becoming tedious.

Resistance is Futile

While game players are unlikely to submit to having large metal probes inserted into the back of their heads (I'd like to see Sony try and sell that to the gaming public), they may well end up being more like The Matrix than Tron. As the power of the games machine begins to free designers to explore new areas of gaming potential, playing will evolve into participation, and that will evoke stronger emotions and reactions.

Examining what psychology can bring to the table as part of the game creation process could be seen as an overindulgence that even the Marquis de Sade would categorize as excessive. However, treated as a practical means by which the game playing experience can be enhanced, a greater understanding of the inner workings of a player's mind will ultimately give the designer more power to create the emotions and experiences that are needed to make the games of the future.

References

Coon, D. 1983, Introduction to Psychology 3rd edition. St.Paul, Minnesota, West Publishing Co.

Gregory, R. L. 1966, Eye and Brain. London: Weidenfeld and Nicholson.

Helmholtz, H. 1909, Wissenschaftliche Abhandlungen, II, pp 764-843.

Hochberg, J. 1978, "Art and Perception". In E.C. Carterette and H. Friedman (Eds.), Handbook of Perception. Vol. 10. London: Academic Press.

Koffka, K. 1935, Principles of Gestalt Psychology. New York: Harcourt Brace.

Marr, D. 1982, Vision. San Francisco: W. H. Freeman.

Miller, G. A. 1956, The Magical Number 7 Plus or Minus 2: Some Limits in Our Capacity For Processing Information. Psycological Review 63: 81-97.

Murdock, B. 1962, "The Serial Position Effect of Free Recall": Journal of Experimental Psychology, 64: 482-488.