“If the user isn’t engaged, the whole experience is a mess (Miles, 2012).” Immersion is a powerful effect for all media. It brings its subjects (i.e. audience, player, reader) into its respective “grand narratives” (Lyotard, 1984, p. 60), while ensuring “the little narrative remains the quintessential form of imaginative invention… (Ibid).” When the subject is experiencing these narratives, they can develop a deep, mental involvement with the media. This strong, psychological connection between media and subject is defined as immersion[1].

One of the most popular contemporary forms of media is that of video games (ESRB, 2010). Video games invoke immersion of players via sensations that resemble rich environments and characters. These sensations, of environment and character, subjectively, language and narrative that is interpreted by the player.

This system of sensation and signification is called semiosis. Semiosis is the systematic process of signs, and their meanings, consciously or subconsciously, in the human psyche[2]. These dynamic systems are apparent in all human societies, and therefore are apparent in contemporary video games. But are developers using this sensation with intent? Can an author or creator truly wield immersion, through semiosis, to their advantage every time?

This essay will explore the use of semiosis, through the narratives’ environment and story of contemporary video games, and their effective immersive experience. Theories from post-structuralists such as Barthes, Lyotard, Deleuze and Guattari will be used to illustrate and define the use of semiosis. It will also include theories from psychologist Csikzentmihalyi. These theories will be focused upon ThatGameCompany’s “Journey,” among other games, to discuss how semiosis is used, and if it can be used by developers effectively.

Post-structuralism is the critiquing of structuralism. Structuralism states society is built upon structures that govern human behaviour via semiosis[3]. More explicitly, between creator and audience. Structuralism states the creator produces a message, based on pre-established structures, and the audience will perceive this message. The audiences’ perception is also based upon structures. Post-structuralist theories explore the destabilizing of these structures exploring the concept that the author’s message to the audience is subordinate and irrelevant because the audience will formulate their own signification regardless of the author’s intended message[4]. Roland Barthes (Barthes, 1967, pp. 5-6), Gilles Deleuze (Deleuze & Guattari, 1994, p. 165), Felix Guattari (Ibid) and Jean-François Lyotard (Lyotard, 1984, pp. 60-61) all support this theory of personal significance, despite the author’s intention.

Flow is a psychological experience that has been theorized by psychologist Mihaly Csikzentmihalyi. Flow is a state of mind that one can enter, and while experiencing this state, have an “optimal experience (Csikszentmihayli M. , 1988, p. 74).”This optimal experience includes aspects such as forgetting about reality and time, extreme focus on the task at hand and producing positive emotions (Ibid). This phenomenon can be mapped in a chart (Figure 1.) to explain the dynamically changing experience of a person.

Figure 1. Anxiety/Boredom Redux Chart illustrating the change in flow experience. From: Csikzentmihalyi, M. (1988) Flow: The Psychology of Optimal Experience

This chart illustrates the dynamic relationship between a person’s skill level, and their challenges. A1 and A4 represent a flow experience when the persons’ skills and challenges are evenly met. Increasing the skill or challenge individually, will result in A3 (anxiety) or A2 (boredom). When the skills and challenges reach a high but equal level, the person will experience the flow sensation again.

Pease and Pease(2004) state that 83% of all sensations the brain experiences are visual. Given the magnitude of information that is received through sight, the environment is the most dominant system of semiosis occurring to the person experiencing it. Virtual environments, like the ones found in video games, act in a similar fashion to real environments. Deleuze and Guattari (1994 pp. 166) conceptualized the theory that whatever sensation a person experiences, they become. This supports the instance when a person experiences an avatar, they become the avatar itself, and therefore the avatar’s environment is the player’s real environment. But the player cannot be immersed by just experiencing the environment itself. The developers, in creating the environment, must construct an environment that immerses the player actively. Developers use semiosis in the form of sensation and signification, to actively immerse the player in the environment. However, this form of semiosis must be succinct in its method. Sensation and signification can be applied to Csikzentmihalyi’s Anxiety/Boredom Redux chart, to show how semiosis can be used to create an immersive environment that the player can experience (Figure 1.a).

Figure 1.a Adopted Csikszentmihayli Model of Boredom/Anxiety Redux illustarting the dynamic relationship between sensation and signification to create an immersive "flow".

This chart illustrates the systemic balance between sensation and signification and its relationship to an immersive flow experience. Sensations are anything a person experiences physically[5]. Signification is any meaning conveyed from a sensation[6]. Points 1 and 4 represent the “flow” experience when sensation and signification are qualified and balanced. Point 2 represents “ambiguity of language,” which is resulted when sensation is abundant (experiences) but not enough signification (myth/belief). Point 3 is the concept “disbelief of myth” which occurs when signification (myth/belief) is abundant, but sensation (experiences) are lacking to support the signification. This dynamic array of sensation and signification can be applied to Journey’s environment.

Journey is a video game that was released in March of 2012 by ThatGameCompany, exclusive to the Playstation 3 console. “Journey is an interactive parable” (Chen, Extended Journey Interview with Jenova Chen - Up At Noon, 2012) where the player assumes the role of an anonymous and androgynous character that explores the “experience [of] a person’s life passage and their intersections with other’s (Ibid).” This game will be used as a case study because it illustrates semiosis effectively, primarily because the game has no text and no discernible speech.

One of the many instances of Journey’s environment, using effective immersion, can be noted in the interaction between the player and the enemies in the game. It is a basic, but effective use of semiosis to communicate that the enemies are attacking, and for what reason. The enemies are composed of two units: the signification and the sensation.

The myth starts with the narrative illustrating to the player that these enemies, or “snakes” known henceforth, are machines built for war (Figure 2). Then the snakes attack the player, tearing their scarf. This sensation and signification qualify each other to create an immersive experience between the player and their environment.

Figure 2. A section of the cinematic that illustrates that the “stone snakes” are machines that are used for war.

To explore the possibilities of the points 2 and 3 on the adopted Csikzentmihalyi model and how Journey might have broken the immersion for the player, the sensations and significations of these snakes would need to be imbalanced.

Point 2, or “Ambiguity of Language,” would be created if a low amount of signification occurred but the sensation was still apparent. If these snakes were to attack the player, and the player was not told why these snakes were attacking, it could bring the player out of immersion because they misinterpreted the language. A player could ask questions like, “Why am I being attacked? Am I doing something wrong?” These questions occur because the sensation was not qualified by the myth.

Point 3, or “Disbelief of Myth,” occurs when the high amount of myth, does not qualify the sensation. If the player was told by the narrative that these snakes are machines built for war, but they did not attack the player, the player would disbelieve the myth. “If these snakes are machines of war, why are they not attacking me? Are they REALLY machines of war?” Because the signification was not qualified by the sensation, it begs the player to question the myth of the game, leading to an emergent experience.

A game that produced a poor immersive experience via “Disbelief of Myth” is Saints Row The Third. Saints Row is a game created by Volition Inc. in 2011, and has some “of the most over the top gameplay experiences (Donovan, 2011).” To understand why Saints Row failed to use effective immersion, like Journey, one must inspect its sensation and myth.

The narrative of Saints Row is constructed around the concept of the virtual world being similar to the real world; at its base mechanics and visuals, Saints Row The Third is a simulator. It simulates a world that behaves and adheres to the physical limitations of the real world. Yet, throughout the game, a player can pick up a weapon in the form of giant fists and punch people until they explode (Figure 3).  

Figure 3. A screenshot from Saints Row The Third demonstrating the sensation of the giant fists is not qualified by the signification that this world is a simulation.

The narrative experienced by the player outlines the limitations in the game. This virtual world simulates the real world and all its constraints. But when the player experiences the sensation of these fists, that narrative immersion is destroyed. Saints Row The Third can be plotted on the Adopted Model of Csikzentmihalyi at point 3, “Disbelief of Myth,” since the signification is not qualified by the sensation. The player has no reason to believe this world is a simulation of the real world. This could break their immersion; i.e. the player might be confused as to what is allowed or not allowed inside the virtual environment. 

Breaking the immersive flow in a game can be done by introducing elements that do not fit the supposed world. One could argue that Journey’s sensations are not qualified by the virtual environment. This is somewhat true. Journey’s world does have simulative sensations that adhere to real world limitations: gravity, light/shadows and movement all behave in a similar fashion respective to the real world. The sand and clouds also exhibit the same properties. So if Journey’s environment is simulating and signifying the real world, why are the sensations still qualified? And by what?

Journey’s use of tropes made its environment familiar, yet alien simultaneously. “It created a world that has its own rules and laws that is on its own a believable system that the audience can relate [to]. (Chen, 2012)” This is the reason that Journey environment didn't need any definition from the developers. An environment that the players experienced and knew precisely what sensations they were experiencing, and why they were experiencing them. The developers used tropes to explain what type of world the player is in, without explicitly stating it. They “[incorporated] representational frameworks from relevant narratives (Salen & Zimmerman, 2004, p. 403)” and elaborated on this framework, twisted it and developed an environment that was familiar yet unique, which “[lead] to [a] compelling game [experience] (Ibid).” The sensations qualified their significations, and vice versa (the player is in a desert because they are experiencing sand; the player is experiencing sand because they are in a desert). This simple, yet effective form of semiosis gave the player the information they needed to know to qualify the sensations they were experiencing. This defined the narrative via the player’s own experience, instead of literal explanations. This use of semiosis can be plotted at point 1 on the Adopted Csikzentmihalyi model because the player has enough sensation and signification to understand their gameplay experience, and can achieve an immersive flow.

The second “little narrative (Lyotard, 1984, p. 60)” that occurs in semiosis is the one that takes place inside the psyche of the person experiencing it. This narrative is internal and can be shaped and affected by factors like the environment, but ultimately rely on the individuals’ signification and connotations (Barthes, 1967, p. 6). If this signification is separate to the author’s intended message or meaning, how can an author ensure their meaning surpasses the audience’s personal connotations? Theoretically, only “a man without history, without biology, without psychology (Ibid)” may interpret the author’s message the way the author intended it. How can immersion be created if the author’s message and the audience’s interpretation are never synchronous?  If the author’s message is subordinate to the signification of the audience, one may question the reason for attempting to deliver an intended message at all. Although the author’s message may be subordinate, it still can affect the audience’s interpretation of their message.

This can be applied to contemporary video games. Developers attempt to create an intended message or signification for the player to interpret, despite the player’s connotation and predispositions. Despite the developer’s message, the player will interpret the game experience as their own narrative (Ibid). Can developer’s create immersion despite the player’s own internal interpretation of the game? This possible immersive narrative experience is dependent on the strictness of the developer’s intended message.

Hypothesize a spectrum of the developer’s anecdotal strictness; the ends of the spectrum consist of the descriptions “broad” and “strict.” For an interpersonal narrative to be immersive, a developer relies on their message to be interpreted by the player to create immersion and understanding. Without understanding or focus, there can be no immersion. “One of the main forces that affects consciousness adversely is psychic disorder (Csikszentmihayli M. , 1988, p. 36)” or, psychic entropy. This psychic entropy occurs when the developer fails to shape the player’s interpretation, resulting in confusion of the narrative. Journey’s narrative can be analysed to see why players did not enter psychic entropy, but rather interpreted the developer’s message and remained in an immersive state.

Journey’s narrative immersed the player because its developer’s intended message was broad. The intention of the developers was to make a narrative that could be interpreted by all players. They achieved this by creating a loose, or broad narrative for the players (Figure 4).

Figure 4. A spectrum showing the placement of Journey’s narrative respective of the developer’s strictness.

In the last part of Journey’s narrative, the sensations and signification are not qualified. This didn’t create an emergent experience, but rather helped the player create their own narrative. The player was experiencing these sensations, however, they interpreted them in their own personal way. These sensations could qualify different myths about the narrative the developers created. This loose narrative was apparent in the scene where the player “dies.” Once the player fails to complete their Journey, they are shown a short cinematic of entities in white cloaks. These entities “resurrect” the player, turning the player an ethereal gold colour, and making their scarf longer than it was previously (Figure 5).

Figure 5. A screenshot from the last area of the game, where the player experiences the sensation of glowing and has unlimited power.

Throughout this cinematic moment, it is not certain that these entities have resurrected the player physically, or spiritually. These sensations are not qualified by any direct significations; this part of the narrative is more ambiguous than that of previous segments thus far. This allows the player to form their own interpretation of what they are experiencing; the player builds their own signification and immersion. The game doesn’t state what myth is occurring. This type of immersive experience is more powerful than narratorial strictness can achieve. This broad message, intended by the developers, has had many interpretations from players, “some people see it as a metaphor of their relationship with their ex-girlfriend. Others see it as a last journey they had with their parent who was recently deceased. And others see [it] as a reminder of their struggle at their work. (Chen, 2012)” The signification that the players built in their psyche, was never challenged at the end of the game; it never confirmed or denied their personal narrative, which allowed them to perceive their personal narrative was the narrative the developers intended. To immerse the player by helping them accumulate their own personal interpretation of Journey’s narrative, the developers used sensations that could be qualified by multiple significations.

If Journey had created a strict narrative where all sensations were explicitly stated and signified by the developers, then the player would have no room for an interpretative experience. If the developers of Journey defined their game as a metaphorical experience (i.e. the subliminal nuances that art can deliver to the audience experiencing it), the player may have a different interpretation. When the player discovers that their interpretation of the author’s message is not the intended one, they will enter psychic entropy, and have a negative experience.

A positive, immersive experience cannot be achieved if the developer’s intended message is delivered through a strict narratorial process. A game that demonstrates an emergent narrative experience is Mass Effect 3. Mass Effect 3 is a game, developed by BioWare, where the player assumes the role of a hero in the future, defending the galaxy against an enemy threat.

By the end of Mass Effect 3, the player has a distinct understanding of the threat, and what steps must be taken to avoid the annihilation of all organic life in the galaxy. This strict narrative, created by the developers, is understood by the player up until the last ten minutes of the game. At the end of the game, players are presented with a narrative that was not presented previously. This narrative explores the idea of creationism and harmony between creator and creation.

Figure 6. The player is greeted with a character, and a narrative plot that has not been presented previously.

The player is then asked, by an entity who, over three games, has chosen to present itself only now, to make a decision that will decide the fate of the galaxy (Figure 6).

Once the player had made their decision, the outcomes were explicitly stated. The developers constructed this strict narrative that gave the player signification that was unable to be interpreted by the players themselves. Concerning strict narrative, developers need “to not [break] the immersion for no reason (Dallas, 2012)”, just to construct the player’s interpretation of their message. This strictness results in psychic entropy and confusion, and an emergent player experience.

One could rectify this narratorial confusion by bringing the narrative to the player’s attention earlier, or less subtly. The developers also may have created an ending where no new narrative plot was introduced, sparing the player from psychic entropy.

To create an immersive experience through semiosis, developers need to consider the “little narratives (Lyotard, 1984, p. 60)” that attribute to a player’s experience and “grand narrative (Ibid)”. These little narratives are composed of the player’s environment and interpretation of the developer’s narrative. To make sure the environment can immerse the player, the creators must apply Deleuze and Guattari’s theories of sensation and must have significations that qualify these sensations. This will help the player understand what they are experiencing, and why. To lead the player to constantly question or guess why they are experiencing something in the environment can create an unwanted, emergent experience.

Giving the player the understanding of the developer’s message is crucial in creating an immersive experience. Understanding and utilizing Barthes’ theology of why the developer’s message is subordinate to the player’s interpretation, can give the player a rewarding, narrative experience that they comprehend. This narrative experience should be broad, and should have sensations that support multiple significations. This allows the player to produce an interpretation of their own and believe it as the intended message. Allowing the player to enter psychic entropy via a diluted or confusing narrative, will only cause an emergent experience.

To use semiosis correctly, in contemporary video games, creates effective immersion for the player to experience. This immersion is a powerful sensation to give the player, giving them a positive experience of the video game. These semiotic theories may not be only applied to video games; they can be applied to almost all forms of media. To experience these narratives and its sensations in books, or movies or music, could help authors, directors and musicians to immerse their audience more effectively than ignoring or using these theories ineffectively.

Semiotics theory is a tool that needs to be succinct in its execution to immerse its subject in its medium. One can note that these theories might be utilized outside of contemporary media, and analysed and applied to other types of dynamic relationships. Work place ethics, teaching and schooling methods, and even psychological exercises, can possibly benefit from using immersion to deliver a more positive experience.

Works Cited

Barthes, R. (1967). Death of the Author. Aspen , 5-6.

Chen, J. (2012). Extended Journey Interview with Jenova Chen - Up At Noon. (G. Miller, Interviewer)

Chen, J. (2012). Immersive Flow. (B. Marr, Interviewer)

Csikzentmihalyi, M. (1988). Flow: The Psychology of Optimal Experience. Cambridge: Cambridge University Press.

Csikzentmihalyi, M. (1988). Optimal Experience: Psychological Studies of Flow in Consciousness. Cambridge: Cambridge University Press.

Dallas, I. (2012). Creating Immersion. (B. Marr, Interviewer)

Deleuze, G., & Guattari, F. (1994). What is Philosophy? New York: Columbia University Press.

Donovan, G. (2011). E3 2011 - Saints Row The Third - Exclusive Interview with Producer Greg Donovan. (Cliff, Interviewer)

ESRB. (2010). How Much Do You Know About Video Games? Retrieved from ESRB: Entertainment Software Rating Board: http://www.esrb.org/about/video-game-industry-statistics.jsp

Lyotard, J.-F. (1984). The Post-Modern Condition: A Report on Knowledge. University of Minnesota.

Miles, W. (2012, October 31). Assassin's Creed 3. (D. Miles, Interviewer)

Pease, A., & Pease, B. (2004). The Definitive Book of Body Language. New York: Random House Publishing Group.

Salen, K., & Zimmerman, E. (2004). Rules of Play: Game Design Fundamentals. Massachusetts: The MIT Press.

Bibliography

Barthes, R. (1967). Death of the Author. Aspen , 5-6.

Chen, J. (2012). Extended Journey Interview with Jenova Chen - Up At Noon. (G. Miller, Interviewer)

Chen, J. (2012). Immersive Flow. (B. Marr, Interviewer)

Csikzentmihalyi, M. (1988). Flow: The Psychology of Optimal Experience. Cambridge: Cambridge University Press.

Csikzentmihalyi, M. (1988). Optimal Experience: Psychological Studies of Flow in Consciousness. Cambridge: Cambridge University Press.

Dallas, I. (2012). Creating Immersion. (B. Marr, Interviewer)

Deleuze, G., & Guattari, F. (1994). What is Philosophy? New York: Columbia University Press.

Donovan, G. (2011). E3 2011 - Saints Row The Third - Exclusive Interview with Producer Greg Donovan. (Cliff, Interviewer)

ESRB. (2010). How Much Do You Know About Video Games? Retrieved from ESRB: Entertainment Software Rating Board: http://www.esrb.org/about/video-game-industry-statistics.jsp

Lyotard, J.-F. (1984). The Post-Modern Condition: A Report on Knowledge. University of Minnesota.

Miles, W. (2012, October 31). Assassin's Creed 3. (D. Miles, Interviewer)

Oxford. (2012). Oxford Dictionaries: Definition of Sensation. Retrieved from Oxford Dictionaries: http://oxforddictionaries.com/definition/english/sensation

Oxford. (2012). Oxford Dictionaries: Definition of Signification. Retrieved from Oxford Dictionaries: http://oxforddictionaries.com/definition/english/signification?q=signification

Oxford. (2012). Oxford Dictionaries: Immersion. Retrieved 2012, from Oxford Dictionaries: http://oxforddictionaries.com/definition/english/immersion

Oxford. (2012). Oxford Dictionaries: Post-structuralism. Retrieved from Oxford Dictionaries: http://oxforddictionaries.com/definition/english/post-structuralism

Oxford. (2012). Oxford Dictionaries: Semiosis. Retrieved from Oxford Dictionaries: http://oxforddictionaries.com/definition/english/semiosis?q=semiosis

Oxford. (2012). Oxford Dictionaries: Structuralism. Retrieved from Oxford Dictionaries: http://oxforddictionaries.com/definition/english/structuralism?q=structuralism

Pease, A., & Pease, B. (2004). The Definitive Book of Body Language. New York: Random House Publishing Group.

Salen, K., & Zimmerman, E. (2004). Rules of Play: Game Design Fundamentals. Massachusetts: The MIT Press.

[1] Oxford Dictionary’s definition of immersion.

[2] Oxford Dictionary’s definition of semiosis.

[3] Oxford Dictionary’s definition of structuralism.

[4] Oxford Dictionary’s definition of post-structuralism.

[5] Oxford Dictionary’s definition of sensation.

[6] Oxford Dictionary’s definition of signification