In this article I have a look at the transliteration of senses and its role in rendering the “feel” of the main menu screen of the Nintento Wii Console
This article is about game feel. It aims to show that gameplay is more than just game mechanics and interaction but the crafting of these as realistic experiences around the limitations and possibilities of the medium being used. The article starts with a brief section on how to render a fictional world into a realistic experience through the use of transliteration of senses. It continues with an analyses of the "opening minutes" of my user experience with the Nintento Wii game console, followed by a very brief conclusion. I'm owing a lot to Michel Chion and Steve Swink. Their works helped me enormously to articulate my thoughts and analyse the experience I had with the Nintendo Wii interface.
The Real and the Rendered
Rendering a fictional world into a realistic experience is a complex process based on the transliteration of senses, that is, the the evoking of senses by means of other senses.
To render something “real” is by no means a new issue: it lies at the heart of communication and has been examined in other media (and in the arts) for centuries.
What exactly are we talking about when we speak of such rendering? Let’s take an example from silent film, brought to attention by an early pionieer of film theory, Bela Balasz:
Shot 1 | Close-up view of a gun’s trigger being pulled |
|---|---|
Shot 2 | A cloud of birds, all the sudden taking off of a tree |
In Shot 2, although no sound is available, we clearly “see” the gun being fired; or to put it more precisely, we “hear-see” it.
In this example, the transliteration of one sense into another (sight into sound) is a realistic rendering of how gunfire “feels”. Yet we must recognize the depth in this rendering: while the sound of gunfire is rendered into the sight of scared birds, the transliteration doesn’t stop here. By means of association, the “sound” we believe to hear is rendered into yet another category of sense, that of touch: we “feel” both its travelling through the air (implied by the reference to sound alone) and the impact of the bullet; we're blinking with the eyes and lean our heads back as if we were trying to escape a slap into the face.
The realness we experience here is "rendered" reality, and by this reality we must understand the realness of the fictional world of the film. We clearly see how this plays a role in immersion, our departure from reality, our crossing into the virtual reality of the film.
This kind of rendering by means of transliteration of senses is also at work during the creation of game feel. I will try to explain how it works by having a look at the opening minutes of our experience with the Nintento Wii game console.
The Main Menu Screen of the Nintento Wii
Love at First Sight (and Sound)
The main menu screen of the Nintento Wii console doesn’t lose a second to set-up the conditions of the feel it wants to create. In our first contact with it (before we start using the controller), it utilizes sight and sought to render this feel.
The first step taken is to utilize the visuals in order to set-up the interface as a three-dimensional space with tangible objects. This tangible ludic realm is rendered by the calculated use of the following design elements:
Design element | Description of how the transliteration works | Type of rendering |
|---|---|---|
Stacked InteractionPanels (A combination of two layers which display menu buttons, date, and time) | The main menu features two interaction panels which are seperated by the use of contours and shading. | Sight to Sight (spatialization) |
Navigation Arrow(s)(Pulsating small-sized arrows to the right and left of the screen) | The arrows are hovering and pulsating above the interaction panels, adding to the depth of the view. | Sight to Sight (spatialization) |
Menu buttons (These resemble TV-screens and display animated images, which is in itself can be considered as a play on the already established visual code and conventions utilized to call the user to break the “fourth wall”.) | The volumetric design of the buttons foster a sight that creates the illusion of three-dimensionality (or depth), something that connotates touch | Sight to Sight &Touch (spatialization and texture) |
Theme Music (a “gentle” melody with high pitch and soft timbre, whose echoeing tunes overlap to create a tingelling, vibrating feel) | The tingleling and vibration of the music stirs up our inscape and stimulatesour sensibility in regard to touch . It opens up an inner space that makes us feel the depth of the "vessel" that our body is. | Sound to Sight &Touch (spatialization and texture) |
In terms of design, the visuals and music are geared towards creating the illusion of depth and stimulating our senses as a first necessary step to "prepare" the user for the experience of a “realistic” feel of controls.
I believe that the musical theme plays a central, embracing role here. The music fosters a hierarchy of senses in which the sense of touching is raised into the dominating figure against a background of all other senses. The musical theme, we could say, *is* about sensuality, is about establishing an “inner” connection with the “hard”ware.
Love at First Motion
Once we start to use the Wii controller, it puts into motion the cursor and other animated graphics. The interplay between these elements as we carry out our actions is designed to enhance the game feel that was hinted at through our first contact with the interface.
Let’s have a closer look:
Design element | Description of how the transliteration works | Type of rendering |
|---|---|---|
Cursor (a stylized hand that leaves traces when being moved) | Moving the cursor leaves a quickly dissappearing trace on the screen, which resembling the feel of touching a surface with our fingertips. | Sight to Touch (trajectory of movement) |
Menu buttons (As described above, but changing in size and contour color when highlighted) | The button animation gives the illusion as if they were three-dimensional objects moving under the impact of a physical force (similar to when we are poking a box). | Sight to Touch ( texture) |
Button Description | When the cursor rests long enough on a button, a button description appears and hovers over the interface | Sight to Sight (spatialization) |
Sound Effect | The only sound effect used in the interface is heard when the button description appears. The sound gives the illusion that the button description seems to “grow” into our sight, whereas in reality it is not animated in that way. It is single frame appearing and disappearing, not an object that closes-up on us and gets buried under the interface panels. | Sound to Sight (trajectory of movement) |
Wii Controller | When hovering over buttons, the controller vibrates in a staccato-like manner, which feels like stumbling over something, or scratching pickles. However, the vibration also makes us recall the sound that is generated when such stumbling or scratching takes place. | Touch to Sound to Touch (trajectory of movement; texture) |
We see that rendering of realistic movement in the Nintento Wii interface is achieved by the agglomeration of various senses through the cooperation of both interface and controls. What fosters the game feel is a synchronization of sight (volumetric indicators & animations), sound effects (attack-decay, timbre), music (tone, pitch, timbre, harmony) and touch (vibration).
In the case of interacting with a button, we see how the synchronization of controller vibration, cursor movement and button animations (or more precisely, the interplay of the transliteration of senses in each one of them) creates the sensation that we “click” something. Clicking a button feels like the slow dissappearing impression of an object that we have touched.
Conclusion
The fun and immersion of gameplay cannot be mere explained by game mechanics or interaction per se. We need to take into account their rendering into a realistic experience. It is this complex process of transliteration of senses and their interplay, often going unnoticed during play, that gives players the game feel that they describe as a fun and immersive (hence realistic) experience.
You May Also Like
