Does it make sense to include neuroscientific methodologies in your games user research? Read this article and find out.
Note: This piece originally appeared on the Bigpoint company blog.
The hype of using neuropsychological methodologies finally arrived in the area of user research, with a particular focus on user testing. One of 2012’s UXCamp Europe presentations was titled “next level of usability testing”, advocating the use of Electroencephalography (EEG), Electromyography (EMG), and Electrodermal activity (EDA) for testing the UX of websites. In this post, I'd like to discuss the additional value of this method for UX research in general, but also with regard to games, the associated costs, as well as clarify the question, whether it makes sense to apply this method.
What does it measure?
So, why bothering to use all these fancy methods anyway? Well, the goal for applying any method is to gain new and deeper insights about our users. And through physiological measurements (called bio feedback these days) we hope to acquire implicit knowledge straight from the unconscious.
Just ask yourself how the term "unconscious" works on you... somehow we tend to overestimate "its power". It's like "we can measure the unconscious", "we know how to influence your mind", "we can control your soul!"... It's not always that exaggerated but you know what I am trying to say. ;)
So to get back on track and be more specific:
EEG measures brainwave activity. So, what does this mean? Well, first of all, it doesn't mean that we measure what the subject is thinking! It just measures the electric activity on a very specific point on the scalp. And to do this correctly, you need a lot of data points (electrodes) that correct the various sources of distortion. The insights we gain from this, is knowing in which part of the brain something may be happening. Triangulating EEG with EMG helps identifying physiological artifacts (confounding peaks from body movement)
EMG measures muscle activity. Here again, electrodes measure the electric activity. As the subject's facial and gestural expressions can be detected, it provides us with insights about the inner state & emotions. Additionally, EMG can be used for eye tracking (although it is not as precise as traditional eye trackers).
EDA measures skin conductance. The underlying concept of this is measuring the skin's electric conductance through its moisture level. High moisture levels are usually associated with a high level of arousel, which can be interpreted (depending on the valence) as either stress or excitement.
What does it cost?
Costs. I asked the speaker of the presentation how much a test like that would cost. For 20 participants, including conduction & analysis, it would be around 20.000 EUR. Pretty expensive if you ask me... If you would like to do it yourself, proper (medical) equipment would cost you about 100 times as much. Yet, there are also a few low-cost EEG devices on the market (called neuro headsets), with fewer electrodes and lower resolution. Measuring EDA is the cheapest of the three methods mentioned, you just need one finger clip connected to an ohmmeter.
Fun Fact: Scientology uses EDA to "measure" how pure your soul is.
Time. Preparation & conduction is more time-consuming than usual usability tests are. Above that, data analysis is much more sophisticated and will cost you (at least) more than twice the time.
Know-how. You need real experts to analyze and interpret the data properly (which will cost you in the end money again). You would wonder how much the data varies between participants! It is not advised to simply read into the topic, if you are a complete newbie. The only thing worse than no data, is false data.
Stress. Your test subjects have to go through some procedures, which can be quite stressful for them. Electrodes must be placed on the participants' bodies and hair, often by using conductive gel. Let me ask you, would you like to have to wash your hair after a study?
Data Validity. You need to spend some time to "train" and calibrate your machines (e.g. for a proper baseline). There are several confounding factors and error sources that may incluence your raw data, while normalizing and filtering it might change its nature, endangering the validity of your data. Above that, test preparation (remember the sticky electrodes) and conduction (strictly controlled lab setting) pose serious consequences to your data's ecological validity.
Will it actually improve results and recommendations?
The data acquired from the above described devices will surely tell us, how the bodies of our participants behave during the test session and how they react to our stimulus material. Let's take a closer look at what we can derive from these methods when applied in games research.
EEG data will tell us, which brain regions are probably associated with processing information of a particular game element. Potential bottlenecks could be identified, which then would provide information architects with insights for a redesign or optimization of a specific part of the game. User researchers could use event-related potentials (ERP's) to identify critical game incidents and do further research for its determinants. This knowledge can be passed on to game designers to create user-centered experiences (literally).
EMG can be used to evaluate game sequences for their elicit emotions (although that is very difficult). More easily, emotional dead spots can be identified, telling which parts of the game appear to be boring and which parts of the recorded video might require special attention. Additionally, EMG is of particular interest in researching and designing motion controls. It provides immediate feedback of the users' movement, which can be used as input for designing game-specific ergonomic control methods. Unfortunately, only very few big developers afford to apply R&D.
EDA is the easiest to measure out of the 3 methods mentioned. It can be used to provide indicators for emotional repsonses at a given state of the game. However, emotions cannot be identified, so its application is of complementary nature only.
As for all of these methods, we cannot be 100 % certain that the collected response is a reaction to the presented stimulus and we cannot make definite implications in terms of "this seemed to be a great moment, therefore repeat this one over & over to improve the game". Meaningful, engaging and fun UX cannot derive from a formula, but is certainly easier to achieve the more "useful" information we collect. It makes great sense to me and is certainly of great aid to have an arousal-valence-timeline for the whole progress of the game so that we can easily look up where exactly we need to change something to improve UX (e.g. on the third tutorial step users get frustrated). However, measuring and interpreting bio feedback data is not that straight forward and needs a lot of investment in terms of time and manpower.
Bottom Line
I do not doubt the significant value of neuroscientific methods for fundamental and clinical research. However, in the domain of UX, the added value, in my opinion, simply does not justify the costs (especially for game developers). Furthermore, in the fast-paced testing cycles of game user research there is just no room for sophisticated analysis. Results have to come fast! And although the prospect deriving from the data appear to be quite promising, we first need to focus much more on fundamental game research before we can apply these kind of methods. As for now, game user research is best advised to stick to traditional UX testing methodologies for producing fast & valuable practical recommendations. However, if you have the chance and time left to play around with new ways of doing research, I would say, go for it! It will certainly give you an extra XP Boost! ;)
Read more about:
BlogsYou May Also Like
.png?width=300&auto=webp&quality=80&disable=upscale)
