User interface design in video games

User interface design in games differs from other UI design because it involves an additional element -- fiction. The fiction involves an avatar of the actual user, or player. The player becomes an invisible, but key element to the story, much like a narrator in a novel or film. This fiction can be directly linked to the UI, partly linked, or not at all. Historically games didn't have any real link to the game's narrative, most likely because early games rarely had strong story elements. Erik Fagerholt and Magnus Lorentzon explored theories of game UI design in their thesis for Chalmers University of Technology titled: Beyond the HUD -- User Interfaces for Increased Player Immersion in FPS Games. They introduce terms for different types of interfaces depending on how linked to the narrative and game geometry they are:

Diegetic

Diegetic user interface elements exist within the game world (fiction and geometry) so the player and avatar can interact with them through visual, audible or haptic means. Well executed diegetic UI elements enhance the narrative experience for the player, providing a more immersive and integrated experience.

Metro 2033 uses a complete Diegetic UI with no HUD elements to help to support the game's narrative. It runs the risk of frustrating the player though slow response times but this forms part of the game mechanic. The character's watch is used to measure how long the filter in the gas mask will last and how visible he is:

Metro 2033

Many games get away with using Diegetic patterns because their narrative is set in the future, where UI overlays in daily life are commonly accepted. This is the case for the latest version of Syndicate. If the story was set in a different time period the UI elements would be probably be considered Spatial (explained in detail later in this post) instead of Diegetic. The DART overlay mode highlights enemies and allows the character and player to see through cover:

Syndicate

Assassin's Creed manages to use a lot diegetic patterns even though it's set in a historical world because the player of the player is using a virtual reality system in the future. So the story is in fact futuristic rather than historical.

The game uses it's eagle vision to highlight enemies and their patrol track. The player and the character see the same thing. There are cases when diegetic UI elements aren't appropriate, either because they aren't legible in the geometry of the game world, or there's a need to break the fiction in order to provide the player with more information than the character should or does know.

Assassin's Creed eagle vision

Meta

Sometimes UI elements don't fit within the geometry of the game world. They can still maintain the game's narrative but sit on the 2D hub plane -- these are called Meta elements. A common example of a Meta UI element is blood the splatters on the screen as a form of health bar, as in Call of Duty: Modern Warfare 2. Blood splashing on the screen within the 2D HUD plane to tell the player that the character is losing health:

Call of Duty

Interacting with the phone in Grand Theft Auto 4 is an interesting example. It mimics the real world interaction -- you hear the phone ringing and there is a delay before the character and player answer it. The actual UI element itself appears on the 2D hub plane though, so it's actually a Meta element, though the start of the interaction is Diegetic. The character is answering the phone but the actual UI element is placed within the 2D HUD plane that only the player sees:

GTA IV

Meta UI elements can be difficult to define in games without a strong narrative element, such as sport or racing games. In Need for Speed: Hot Pursuit I still feel that the speedometer in the 2D HUD is meta because the player's character -- the driver, would know what speed the car is is traveling at and therefor it forms part of the narrative, as much as there is one. Other HUD elements such as track position or track location can be more difficult to define, some could be considered Meta because the drive would have that information while the rest are simply Non-Diegetic. The HUD elements such as the speedometer to relay information about the car to the player:

Hot Pursuit

Spatial

Spatial UI elements are used when there's a need to break the narrative in order to provide more information to the player than the character should be aware of. They still sit within the geometry of the game's environment to help immerse the player and prevent them from having to break the experience by jumping to menu screens. The closer these follow the rules of the game's fiction the more they can help immerse the player.

Splinter Cell Conviction also adopts Spatial elements in the form of projections that illustrate objectives within the game world. Their scale does seem to challenge the fiction slightly more than other examples. Type is overlaid in to the environment to communicate messages to the player rather than the character:

Splinter Cell Conviction

Fable 3 is another example where Spatial elements are used to provide more information to the player and prevent them from jumping to a map screen. The glowing trail almost fits within the fiction given it's magic aesthetic quality but the character isn't meant to be aware of it. It guides the player to the next objective. The sparkling trail allows the player to guide the character in the right direction:

Fable 3

Spatial elements can be beautiful pieces when they work with the geometry of the world. These Spatial elements from Forza 4 demonstrate a simple style can contrast the rich 3D qualities of the game. Bold iconography combined with strong typographic layouts help establish a beautiful art direction for Forza 4's UI:

forza4

Non-diegetic

Then there are traditional non-diegetic elements, these elements have the freedom to be completely removed from the game's fiction and geometry and can adopt their own visual treatment, though often influenced by the game's art direction. I think these elements are best used when the diegetic, meta and spatial forms provide restrictions that break the seamlessness, consistency or legibly of the UI element. World of Warcraft uses a mostly Non-diegetic UI, one exception being the Spatial player names. It allows the user to completely customise it, hopefully ensuring a familiar experience. Most of the UI elements in World of Warcraft sit on the 2D hub plane, some elements sit within the world's geometry such as the player names however the character isn't aware of any of the UI:

wow

Finally, Mass Effect 3 uses many Non-diegetic UI elements in order to inform the player of the character's selected weapon and power -- among other things. Given it's futuristic setting I can't help to think if some of this information could have been integrated in to the game world, narrative, or even both. The Non-diegetic elements still inherit the visual style associated with the game world:

Mass Effect 3

Conclusion

Physical interaction methods and immersive technology such as VR headsets promise to challenge game UI design, allowing for a stronger connection between the avatar and character as both engage in similar actions at the same time. Technology provides an opportunity for deeper levels of interaction with the addition of audio and haptic elements. This will mean less use of non-diegetic UI.

Game UI has a key advantage (or disadvantage from some perspectives) in that players are often engaged with the narrative and/or game mechanic enough for them to learn new interaction patterns, or forgive bad ones. This is likely the reason so many games have bad UI, as testing needs to encompass the core game mechanic while UI is seen as secondary.

Credits: 3D icon design by Nicolas Morand.