The cognitive science behind games user research

Introduction

When conducting Games User Research at Sprung Studios, whether it be usability testing or expert reviews, we often talk about the idea of the research being “based on Cognitive Science principles”, but what exactly does that mean?

Client reports are rarely the right time or place to explain the Cognitive Science principles that underlie games user research, or how video games and the brain are related. This can be problematic, especially in the case of expert reviews, as it can unfortunately lead to the false impression that the work is simply the opinions of the researchers, or a subjective catalog of what the researchers did and didn’t like about the game.

Therefore, this article has two goals:

Cognitive Functions

Cognitive Science is an interdisciplinary field that is concerned with understanding the various systems of the brain, the cognitive functions involved, and the processes that govern their operation. Throughout this article we’ll be highlighting a few of these cognitive functions.

This is not an exhaustive list, and it is important to understand that the brain does not work in distinct buckets. These functions often overlap, and a problem in a game can be rooted in multiple cognitive functions. The simplified explanations presented below are simply to teach the basic ideas.

Attention

Attentional Spotlight

While a player can see the entire screen, a common myth is that they can also pay attention to the entire screen at the same time. However, that is generally not true as attention acts more like a spotlight that can be directed.

When it comes to game design, this means that 1) it’s not safe to assume that a player will notice something on the edge of the screen just because it is visually present, and 2) work needs to be done to guide the player’s attention to the part of the screen you want them to focus on. Failure to do so is one of the main reasons that players miss important on-screen events.

Endogenous vs. Exogenous

Guiding of attention can happen either voluntarily or automatically.

Voluntary shifting of attention is called endogenous control and commonly seen in games like Hidden Folks (think: Where’s Waldo?), where the player is actively searching for a target item.

Automatic/reflexive attention is called exogenous control, and is the reason why notification pips, popups dialogs, and flashing animations draw the player’s attention.

When designing a game, consider which type of attention you’re aiming to achieve. For example, consistent iconography between a map and a key helps to maximize endogenous control, whereas you may want to limit popup notifications when the player is in combat to avoid automatically/reflexively drawing their attention away from the action.

Useful Field of View (UFOV)

When looking at any part of the screen, there is a limited area around that point where information can be effectively retrieved. This is called the Useful Field of View (UFOV). The size of the UFOV does not span the entire screen, so placing important information on opposite sides of the screen can be problematic, especially if the player is using a widescreen display.

Additionally, the size of the UFOV is heavily dependent on age (it gets smaller as you get older), so it’s important to consider the target demographic when considering where to place UI elements on the screen.

In game design, this comes up frequently when displaying off-screen markers. For example, in Assassin’s Creed Valhalla, if players are looking at the off-screen markers on the left side of the screen, it will be difficult for them to simultaneously see the one of the right side without moving their eyes:

Iron Space gets around this problem by moving enemy markers into a radar that is placed more centrally:

The same idea applies to larger UI elements. Many first and third person shooters place health bars and ammo count at the edges and corners of the screen. The Division 2 places that information closer to the center of the screen so it’s easier to access with a quick eye movement:

Inattentional Blindness

Inattentional blindness is the inability to see something that is in full view of the screen due to engagement in another task. The classic example of this effect comes from the paper Gorillas in Our Midst: Sustained Inattentional Blindness for Dynamic Events (1999).

This effect comes up time and time again in games. Imagine fighting a raid boss in World of Warcraft and seeing your party members die because they’re standing in some area-of-effect damage. Sound familiar? This is inattentionalblindness in action and is usually not the player’s fault, but instead the result of the game drawing the player’s attention to the wrong thing.

This also comes up frequently with in-game tutorials. In Assassin’s Creed Valhalla, tutorials can pop up in the middle of combat. Failure to see these tutorial boxes due to focusing on combat can have long-term consequences for the player’s comprehension of game mechanics:

Perception

Gestalt Principles

Gestalt Psychologists outlined a number of principles of perceptual organization and grouping. They help to explain why certain elements appear to belong to the same group, while others do not. This is particularly useful in UI and motion design because it allows you to perceptually group items together without literally drawing a box around them.

A few examples:

Breaking these principles can lead to confusion for players. For example, in the Asphalt 9: Legends example below it’s unclear what the +12 refers to because it’s not placed in close proximity to any other element:

Memory

Procedural Memory

Not all memories are encoded actively with conscious awareness. Procedural memories are implicit, skill-based, and learned through repetition. One real-life example of this is learning how to tie shoelaces; with enough practice and repetition, it becomes automatic and you can do it without thinking.

In the context of game design, one place where procedural memories come up is with control inputs. Over time, players develop a memory for certain controls, and is one of the main reasons that games offer settings for joystick and mouse sensitivity; they allow players to tweak input settings to match what they have stored procedurally. Similarly, some players have developed a strong procedural memory for playing with inverted-Y controls, and games become almost unplayable for them if no inverted option is available.

Another common issue is the use of non-standard key bindings and/or the lack of key remapping options. Non-standard bindings, like in the case of Undertale, make it very easy for players to press the wrong button when performing common actions:

Short-Term Memory

The Magical Number Seven, Plus or Minus Two (1956) is one of the most famous papers in Cognitive Science, and states that short-term memory has a limited capacity of 7 items that can be effectively held in memory at a time. However, that value has been disputed and more recent work suggests the limit is closer to 4 items.

This memory capacity limit is often violated in video game tutorials. In New World, for example, tutorial text is used to explain key modes in the game. However, the amount of information being presented is too much to be contained within memory without re-reading. As a result, the information becomes more difficult to learn and is less likely to be remembered by the player:

Serial Position Effect

This is more commonly referred to as primacy and recency effects, and is the idea that items at the beginning and end of lists are better remembered, while items in the middle tend to have the worst recall.

These effects come up frequently in game design, and any time there is a list of information that the player needs to remember it is important to consider how those list items are arranged and ordered. In the New World example above, the length of the list makes serial position effects stronger, and the bullet points in the middle might not be remembered as well as the ones at the beginning or end.

It is important to note that these effects apply even if the “list” does not strictly look like a bullet-pointed list of items. For example, games with long guided tours that explain what different UI buttons do, like in the case of Stormbound, can also exhibit these effects. This is one of the reasons why teaching within the context of gameplay is often more effective; memory is contextualized and reinforced with actions. Without this, teaching becomes more susceptible to serial position effects and players may forget explanations of certain features:

Spatial Processing

Navigation

There are many different forms of spatial processing, but in general this refers to the way humans generate an understanding of 2D and 3D objects, as well as the relationships between them (e.g., distance, rotation). One place where spatial processing is heavily utilized is in map reading and navigation.

When navigating the world, humans encode spatial information using different reference frames, the most common being allocentric and egocentric:

In games, the readability and learnability of maps is drastically improved when players use their preferred navigation strategy. However, some games force one particular reference frame (usually landmarks via map icons), without the option of using the other.

The Division 2 is a good example of a game that includes both types of navigation to suit a wider audience of players, with the option to turn the different aids on or off:

Route-based/egocentric navigation does not necessarily need to take the form of an explicit route line drawn on the map. It could also be a simple compass to show the direction/heading of key features, like in the case of Call of Duty: Warzone:

Executive Function

Executive functions are a class of cognitive systems that integrate information from lower level processes like vision or auditory processing, and form the basis of common functions like decision-making and planning.

They can be thought of as being at the top of the cognitive hierarchy; as such, unstable foundations can cause a domino effect that negatively impacts the performance of these executive functions. For example, low contrast text in a UI is first and foremost a vision issue, but this can cause additional issues later for the ability to ignore distractors at higher levels of processing.

When it comes to video games and the brain, how well executive functions work can depend on many factors. Age is one example, as executive processes are not fully developed until late adolescence, and begin to decline during older adulthood. As such, effective game design requires an understanding of the game’s target demographic. High amounts of cognitive load can also have a detrimental effect on executive function, which leads to the question of how games and UIs can be designed to minimize load and improve the performance of these cognitive systems.

Information Updating

The brain needs to manipulate, revise, and update information, and a lot of this relies on working memory. Working memory acts a bit like scrap paper; it’s a memory location that’s used to store information that you’re actively thinking about and working with. Unlike short-term memory, there is a very strong emphasis on the active manipulation of information, thus placing it in the domain of executive function, rather than a simple memory store.

Working memory has a special importance in game design, as there are many occasions where the player needs to actively work with information for the purposes of decision-making, for example.

Part of the issue here is that working memory has a limited capacity that is heavily impacted by cognitive load. A UI that is poorly laid out, or doesn’t present enough information, or presents too much information can reduce the effectiveness of working memory. An effective UI design, then, should strive to offload as much of the work from the player as possible so they don’t need to store as much information in their heads.

We see this issue crop up in collectible card games like Hearthstone. When players build custom card decks, one of the things they need to consider is the interactions between their cards, part of which is determined by the status effects offered by the card (e.g., Silence and Battlecry in the example below). Given the large number of cards available, it becomes difficult for the player to keep all of these potential card interactions in mind when building a deck. This is hindered by the low contrast of the status name text, the lack of status icons to aid quick recognition, and the inability to filter these statuses without relying on a text search (new players might not know or remember what to search for):

Task Shifting

When working with multiple tasks or multiple sets of information, the brain needs a way to flexibly switch back and forth between them. This process could involve 2 completely different tasks, or more likely, 2 distinct but overlapping tasks, in which case working memory involvement is required. In the latter case, it is especially important to consider how much relevant information is carried over between the 2 tasks to reduce how often task shifting needs to be performed.

Task shifting comes up frequently in games, especially in MMORPGs that contain many complex systems. For example, most MMORPGs require both quest reading and map reading. There is usually a quest list with a currently active quest that the player needs to understand, as well as a functionally distinct but overlapping system that helps the player navigate to the quest location.

In Guild Wars 2, these 2 tasks are linked together using consistent colors and iconography in a way that reduces cognitive load. The same colors and icons are used across both tasks, with quest names presented on the map itself. Just enough context is carried over between the 2 tasks so the player doesn’t need to constantly shift between them or feel overwhelmed. This is a good example of the most barebones way to handle task shifting, while still remaining effective:

Inhibitory Control

The brain needs a way to actively ignore irrelevant or distracting information, and this is the role of inhibitory control. While you may sometimes want to explicitly draw the player’s attention toward something important (e.g., using popups, notification pips), there are also many times where you want to present information without it being distracting.

Inhibitory control allows a player to ignore those potentially distracting elements. However, the problem is that inhibitory control, and the ability for a player to exert control over their own attention, exists on a spectrum; some players are better at it than others. As a result, it can be easy to accidentally pull a player’s attention away from something important. What can be done to improve usability for those players?

A few ways to handle this are to make use of reduced contrast, or increased transparency, to limit how distracting unattended UI elements might be. But this is often a difficult balance to strike because making something too subtle could lead to inattentional blindness.

Another approach to this problem is to use state-based visibility of UI elements. As an example, receiving private messages when in the middle of a ranked PVP match can be very distracting for some players. Guild Wars 2 solves this by providing an option to hide the Chat Panel when the player is in combat:

Inhibitory control uses attentional resources, which have a limited capacity. If a screen contains too many elements that draw attention, it doesn’t matter how good the player’s inhibitory control is, those resources will eventually be drained and they will become more susceptible to distraction as a result.

This is often seen in casino games, like Slotomania, which contain many visual animations, sound effects, and other attention-grabbing elements. As a result, the player’s ability to control their own attention diminishes over time:

As an interesting side note, the rise in popularity of location-based augmented reality games (e.g., Pokemon GO, Orna) presents an interesting game design challenge when considering inhibitory control. Research shows that inhibitory control is sharply diminished when performing concurrent physical activities like walking. As such, it is important to consider the context in which the game will be played and how that might inadvertently impact usability.

Final Word

We hope this brief crash course in Cognitive Science has provided useful insight into the types of principles that underlie Games User Research. More importantly, we hope it is now easier to understand why usability testing and expert reviews are not simply the opinion of the researchers, but rather that they are rooted in academic studies of different brain functions.

If you’d like to learn more about the relationship between Cognitive Science and Games User Research, we recommend reading The Gamer’s Brain; there are also many useful videos discussing how video games and the brain are related. For more UXR content like this look into our previous article on How to Conduct User Research Without Users and stay tuned to our News Page.

Sprung Studios is the world’s leading UX/UI design partner for games, check out our homepage for more information on our services.