Cheaper physiological sensors lead to an increased use in the game industry. This post explores past and present applications of biofeedback game interaction and psychophysiological evaluation in game user research.
(This post originally appeared at the physiological computing blog)
I have been deeply interested in physiological game interaction and psychophysiological game evaluation for at least the past five years, spanning my PhD and postdoc time, several presentations for research institutions and game companies, a growing list of publications, and other articles. In the meantime, physiological sensors have become much cheaper and today we are seeing companies like Neurosky and Emotiv with low-cost physiological sensor products reaching a large amount of customers. My colleague Mike Ambinder at Valve is now even looking into applications of biofeedback input for commercial game titles (PDF) some of this was demonstrated at GDC 2011. So, this is definitely an exciting field to work in. For the rest of this article (which reproduces parts of my workshop paper), I will recapture my CHI workshop talk and discuss some of the applications for game interaction and game evaluation from a Physiological Computing side.
Digital games are an excellent field of application to explore the pros and cons of interacting with physiological sensors, because the consequence of failure are less severe than in critical computing systems, where life or death depends on the right algorithm. This is one of the reasons why physiological computing and psychophysiological game evaluation have become so popular in the game user research community. So, what applications of physiological sensors have we seen in the past and which ones are we likely to see in the future?
Since the 80s, people have tested physiological input in biofeedback games. If you look at the current research body and the current biofeedback applications that you can buy on Amazon and the like, you will find that many of those software applications are actually biofeedback games. So, what is a biofeedback game you ask (or you might not if you read this blog regularly). Those are games developed to make us more aware of our physiological state and train us to control it. For doing this, manufacturers usually use games, as the challenges, progression, theme, and rewards in a game are usually enough to get us excited about our bodily states. One could even argue (using a controversial neologism) that biofeedback is the gamification of physiological control.
The biofeedback equipment manufacturer Thought Technology Ltd. was one of the first players on the market to investigate physiological input with a modified GSR2 sensor and the racing game CalmPrix in 1984 (see Kiel’s post related to this). Around the same time, Atari developed—but never officially released—the Mindlink, an electromyography headband based on an earlier medical plugin product called the Atari Bionics system. Since many test players got headaches from moving their eyebrows, the product was cancelled.
Later physiological games that were developed for a larger market were, for example, a Nintendo 64 electrocardiographic (ECG) biosensor included with the Japanese version of Tetris 64 and a galvanic skin response (GSR) called Lightstone developed for the game “The Journey to Wild Divine,” which was moderately successful. While the Wild Divine biofeedback game trained users to control their physiological activation—ultimately with the goal to help them relax—Tetris 64 would adapt the game speed based on the user’s heart rate (HR) with two different speed modes.
Recently, two development trends are visible for physiological game interaction.
Biofeedback and health games such as EA Sports Active 2 tend to use pulse oximetry and/or HR sensors training users to control their physiological state. Examples are the Nintendo Wii Vitality sensor and the Ubisoft Innergy sensor—both pulse oximeters—which are planned to be shipped with relaxation games.
Electroencephalographic (EEG) systems use neural signals to fully control or augment game interaction. Examples are affordable EEG interface solutions that are specifically targeting game interaction, such as OCZ Neural Impulse Actuator (NIA), Emotiv EPOC, or the Neurosky Mindset. The latter ships with a demo game called “The Adventures of NeuroBoy”, where the player has to use telekinetic powers to push, pull, lift, or burn objects using the EEG MindSet.
One of the first research prototypes for a biofeedback game was developed at NASA based on a simulator system that used EEG to control the automation level of a simulated flight. Biofeedback changes were not explicitly shown, but implicitly felt by subjects operating the system, thus exerting control over what Steve calls the biocybernetic loop in physiological computer games. What does this mean? At each game event point, the game system receives physiological data from the player to control how the game world responds. Ideally, this response is tailor-made for an individual player based on testing how players respond bodily to game events. Therefore, the game becomes adaptive to the player.
Kiel discussed the potential cheating issues in a biofeedback game called Tokimeki Memorial Oshiete Your Heart in his first DiGRA paper on affective gaming. In the game, players could cheat by jogging before play to increase their success with a virtual animated date. Kiel went on to define affective gaming as using the player’s emotional state to manipulate gameplay variables. This means distinguishing user emotions, such as at-game (e.g., physical failures) or in-game frustration (e.g., misunderstanding) . In general, Kiel’s and Steve’s past papers on affective gaming give you a good overview of the ideas that were pushing the field.
Games developed in biofeedback research include among others: a 2D side-scrolling game with many direct and indirect physiological sensor controls augmenting game controls, a first-person shooter game using implicit and explicit biofeedback, games that use relaxation as a winning condition, modifications of World of Warcraft to control shapeshifting via alpha brainwaves and Half-Life 2 mods where GSR controlled some gameplay variables.
Game user research has always borrowed from psychological methodology and more recently we have seen an increased interest especially in research in using psychophysiological sensors for measuring factors of emotional engagement.
Using physiological sensors to evaluate game engagement requires much work and expertise from the game user researcher. As Steve recently outlined on this blog, there are many pitfalls for researchers when using sensitive physiological measures to evaluate gameplay. In this type of research, the game user researcher usually follows an experimental psychology approach that is characterized by 4 features:
Comparing controlled conditions
Importance of statistical power (right tests for measures)
Controlled participant sample
Counterbalanced design (to remove order effects)
Physiological data is sensitive, variable and difficult to interpret without a high level of experimental control.Therefore, we are at risk to oversimplify some interpretations of physiological data and do not keep in mind its one-to-many relationship to psychological effects. For example, galvanic skin response is used too often to infer emotional qualities although it is a highly ambiguous measure regarding emotional labels.
In general, these difficulties regarding proper experimentation with and interpretation of physiological measures mean that equipment and time costs (such as researcher and participant time) often outweigh the non-established benefits of these methods over traditional user experience methodology (such as questionnaires and interviews or sometimes eye tracking). Research personnel needs to be trained extensively in interpreting (and correlating) physiological metrics (with psychometric measures). However, some third-party contractor labs, such as Immersyve, Bunnyfoot, and Vertical Slice are vocal in advertising the benefits of various physiological metrics for the right research questions. Unfortunately, there is currently no gold standard of methodologies, analysis or visualization tools. Unless, the problem of “easy interpretability” is solved, these measures will likely not become common to game user research. A good starting point for game user researchers unfamiliar with physiological measures is for example Ben’s recent Gamasutra article based on psychophysiological research methods of recent years, explaining some physiological measures and how they can be applied for testing games. Another starting point is this panel presentation I organized a few years back for GDC and Future Play.
The study of emotional and engaging experiences in video games has become more popular over the last decade. Common physiological measures are facial electromyography (EMG) for assessing player emotions, cardiovascular measures such as interbeat intervals, GSR, and only sparsely EEG because of its complex analysis procedure. Matias Kivikangas and colleagues have recently provided a good overview of the current state of psychophysiological game research. Common approaches have emerged from this previous work, distinguishing physiological analysis on a temporal dimension: Studying phasic psychophysiological and behavioral responses at game events (points in time, or the micro level as Steve calls it) and studying tonic responses to variations in game variables (time span or the macro level) .
One problem game research has to solve is making the interpretation of this data meaningful in terms of facilitating design decisions for developers. A first step in this direction has been taken in Regan’s emotional interpretation of EMG and GSR data using fuzzy logic. More steps in this direction are necessary to facilitate the interpretation of these large datasets, possibly creating visual aids for faster navigation and easier interpretation of physiological game engagement. This is where I see much research and industry collaboration potential in the next years.
Read more about:
BlogsYou May Also Like
