Better, Stronger, Faster: How Games Will Change What We’re Capable Of

This and other posts from this author can be found at: www.motivateplay.com

For a few years now I’ve been interested in the idea of cybernetic systems – that is, systems of information communication and control that typically include a human controller, a machine enhancement, and regulatory feedback between the two.  For me the most interesting part of such systems is their ability to augment and amplify basic capacities of the user.  While the earliest human-machine systems expanded the physical feats that could be accomplished by their controllers,  later systems, particularly with the rise of computing technologies, have come to extend our basic cognitive capacities – those for attention, perception, memory storage and retrieval, and, of course, computation.

Present technologies and our interactions with them, however, are changing rapidly.   Our devices – particularly media and communication technologies – are becoming increasingly ubiquitous, mobile, and interconnected.  Further, they are increasingly incorporating sensor technologies for presenting us with personally-relevant, actionable information.  Considering these trends, I’d suggest that we are at the cusp of a revolution in man-machine systems, one which will amplify not only capacities for things like movement, perception, and memory, but for motivation and problem-solving.  And further, I’d argue that gaming – more so than any other type of domain or technology – will be what drives the development and mainstreaming of such systems.

Cybernetics – Foundational Principles

Cybernetics as a theoretical field arose in the late 1940s with the work of math prodigy Norbert Wiener.  Heavily influenced by earlier research into the nature of physiological homeostasis and neuroscience, as well as the recent arrival of the electronic computer, Wiener was interested in automation and the idea of “thinking machines.”  In approaching these topics, Wiener concerned himself with the nature of looping inputs – that is, how information is sensed (inputted), compared (processed), and then re-inputted into the same initial function.

Broadly speaking, cybernetics is a systems approach to understanding interactive or reactive parties and their relationships.  However, Wiener’s work centered on human-machine interactive systems in particular.  This was due to his professional focus at the time – anti-aircraft weapon systems.  Wiener had noted that operating such guns required a great deal of computation prior to actually firing the weapon.  This computational effort on part of the human soldier was taxing, hindering his ability to calibrate a successful shot.  Wiener understood this situation in terms of a complex system, one in which soldier and machine each played a role.  As a solution to the problem, he proposed shifting the computational effort to the machine element of the system, which could more quickly calculate and provide estimates on aircraft trajectory.  In turn, the human soldier would be permitted a greater amount of time to successfully aim and fire a weapon.

Key to this process was the idea of negative feedback.  Based on his knowledge of human physiology, Wiener understood negative feedback as serving a regulatory function for a system. – in human bodies, specific homeostats regulate physiological conditions so as to achieve and maintain desirable bodily states (e.g., temperature, respiratory rate, etc.).  Wiener believed that additional homeostats might be constructed that exist external to the human body, capable of gauging the differential between current and desired states for other types of performance.  It was this thinking that guided his conception of the cybernetic system – a system in which users could extend themselves beyond their physical boundaries through the use of machine components, with regulatory feedback sent from machine to human in order to steer behavior and subsequent input.

So Where Are All the Cyborgs?

While Wiener’s work went on to influence a variety of fields, ranging from communication theory to neuroscience to medicine to electrical engineering, it is science fiction and not science proper that introduces most people to the concept of a cybernetic being.   RoboCop, Steve Austin, replicants, Cylons – these are the examples that typically come to mind.  Yet, there are certainly less extreme real-life instances of cybernetic beings that we might note.  For instance, there are the physically impaired, such as veterans or others suffering from chronic disabilities, who make use of prosthetic limbs and sensory implants [however, compared to their science fiction counterparts, these examples might come off as a bit less enchanting – though unquestionably amazing feats of technology, these devices typically serve to restore natural human abilities rather than enhance them].   Then there are also the few ultra-committed pioneers of wearable computing, like Thad Starner and Steve Mann – individuals who have actively integrated computing technologies into their person in order to enhance their abilities to calculate and communicate in everyday interactions.

In other words, real world “cyborgs” have until now mostly consisted of marginal groups. As such, the situation is comparable to that of virtual reality 20 years ago.  Back then, while the concept was a sci-fi staple, very few people in grounded reality were actually inhabiting VR.  Interestingly, Edward Castronova has since pointed out that despite all the promise and hype about sensory-immersive VR, it ended up being the social presence and commercial budgets of some key virtual worlds that moved VR from science fiction to the commonplace.  MMOs, not helmets.

Similarly, I’d predict that it will be games that bring about the mainstreaming of everyday cybernetic technologies (albeit, as was the case for VR, in forms a bit different than what science fiction has lead us to anticipate). Though the shrinking sizes and price tags of computing tech and the growing preponderance of sensors will certainly contribute to this, games will be what permit man-machine systems to be broadly and widely psychologically engaging and commercially viable.  This is because games 1) can calibrate our intuitions about the real world, 2) will drive the development of software that adapts to user emotional states, and 3) can transform boring data into motivationally relevant information.

Games Calibrate Our Intuitions About the Way Things Work

Video games are systems of controlled behavior, in which user input is modified based on feedback in order to reach desired goal states.  Within games, human players engage a virtual environment, communicating with it so as to learn how in-game mechanics operate. The player sends input into the system, in response to which he or she is greeted with visual, audio, and sometimes tactile feedback on their performance on a given task.  Video gameplay is cybernetic in that, through the repetition of trial and error of different strategies, the machine helps enhance the player’s ability to solve the puzzles and obstacles inherent to the task at hand.  It just so happens that the task at hand is within the machine itself.

This cybernetic nature of gameplay is not lost on designers like Will Wright.  Back in a 2007 TED talk, sporting a home-made cyborg outfit, Wright explained how he views games not merely as play environments, but as special toys that can hone and tune particular skills.  While demo’ing the then-upcoming Spore, he suggested that, just as a telescope can augment our sense of sight, computer simulations can recalibrate and re-map our intuitions across vast scales of both space and time.  Wright’s point was that game designs (particularly those like his) can allow players to not only build worlds in their imagination, but to also extract that imagination into physical form through easy, fun simulation tools.  In this manner, he argues, games – particularly those that include a relatively large “possibility space” for the player to explore and test –  can amplify our capacity for imagination and creativity.

And indeed, this is the premise essentially underlying Jane McGonigal’s more recent efforts with game experiences like EVOKE. If the possibility space of a simulation is made to reflect certain real world conditions, it permits the user a chance to critically examine and engagingly play with a situation he or she might not regularly access.  What’s more, the feedback users receive may then be useful in generating actions and decisions in the real-world counterparts of the simulation.

Of course, the obvious issue becomes the fidelity of the simulation – that is, how accurately the real world is mapped into the virtual representation.  In their simulations, games implicitly make arguments about the way the real world works (this is the basic idea of persuasive games) – yet who is to say the model offered isn’t biased towards the perspective of a given designer?  And in addition to infidelity by intention, there is the possibility of infidelity due to scope.  For example, Spore eventually came and went, catching a lot of flack for offering a game experience that didn’t truly give players the chance to inhabit the role of a not-so-blind watchmaker.  But honestly, any game, even one as ambitious as Spore, would be hard-pressed to offer a possibility space sufficiently expansive to calibrate intuitions about things like evolutionary systems and still consist of meaningful feedback on player decisions.

Such an approach, however, could still certainly be useful for games meant to simulate systems less encompassing than the entirety of biological and cultural evolution.  For instance, games like Foldit and EteRNA focus on much more specific biological phenomena – protein and RNA folding structures, respectively.  Each game uses puzzles to teach the user about potential structural patterns, then challenges these players to come up with new designs, some of which are then actually synthesized and tested in real world labs.

Games such as Foldit and McGonigal’s show that, though they use virtual settings to calibrate the intuitions of players, games can give users the appropriate feedback for contributing creative solutions to real world problems and puzzles.  The proliferation of games like these – in which simulation-based feedback can influence real world practices – is just one way in which gaming will contribute to the mainstreaming of cybernetic systems.

New Games Will Accommodate User States

Again, key to the cybernetic relationship is the use of regulatory feedback to minimize the difference between current and target performance.  Typically, this means using machine feedback to better fit performance to task demand (tasks which, as discussed above, may be real or virtual).  However, though not what Wiener originally conceived, an alternate approach could just as well be to employ feedback to better fit task demand to user performance.

This is exactly the approach taken in physiological computing, a subfield of HCI that recognizes communication to be a two-way street, one in which machines, not just humans, can behave adaptively and proactively.  Such designs rely on a “biocybernetic loop”, a two-stage process in which the system first uses physiological measures to gauge the user’s psychological state and then , if flagging this state as undesirable, adjusts the nature of the task accordingly.  For example, a program may detect that a user is frustrated with a certain software task, at which point it may offer a helpful hint.  Or, if it detected the user to be bored, it might increase the complexity of the task. 

Adaptive computing designs will be most useful in cases where the emotional engagement of the user is a primary concern.  For example, auto-pilot systems and other programs in which it’d be nice to amplify or down-tune user attention and emotional investment as needed.  However, another domain in which the relevance (and market potential) of such designs is obvious is gaming. [Note - I should point out that Gamasutra Expert Blogger Lennart Nacke is actually a leader in this emerging field, and has recently posted an excellent summary of physiological computing and its implications for game research and design.]

Granted, the idea of dynamic difficulty adjustment in games is not a new one – many different designs have attempted to incorporate user performance into setting task challenge, so as to increase the chance of “optimal experience”.  However, whereas most methods of DDA currently rely on in-game performance metrics as proxies for skill and ability, physiological computing relies on correlates of the user ‘s emotional state, which is what ultimately matters when designing for user enjoyment.  Further, while most DDA designs operate in the time domain of rounds and levels, physiological gaming can monitor user states in real-time, allowing for a much more granular detection of which precise elements of a task are eliciting a given response.  As such, physiological computing may open the door for a whole new level of dynamic, engaging game designs.

So, just as games can provide feedback to calibrate user skills and intuitions, users may soon come to regularly provide feedback that calibrates game challenges.  In this sense, the future of gameplay may be real-time cybernetic systems with two separate, overlapping channels of regulatory feedback.

Sensor Data + Game Context = Behavior Change

Though they certainly include human-machine feedback loops, the examples discussed so far are still basically a user sitting at a screen playing a game.  That is,  images they stir up don’t quite resemble RoboCop or the Borg.  However, another growing trend – the emergence of cheap, mobile, and simple sensing technologies – may bring us a step closer to such beings.

In addition to the variety of devices that can be worn on different parts of the body, sensors have crept their way into our homes, cars, phones, and various other appliances.  As a result, an individual that is so inclined can be presented with data on almost any status or behavior metric imaginable – location, sleep patterns, caloric intake, driving habits, productivity, level of happiness, etc..  In this manner, sensors are already augmenting our basic capacities for perception and attention, allowing us to reflect on feedback information otherwise beyond our level of conscious awareness.  What’s more, as a recent article from Wired notes, for those that actively collect and process this data, it can offer real insights into one’s habits and lifestyle.

The Quantified Self movement is comprised of just such people.  Ranging from the mildly curious to the fanatically narcissistic, these individuals track themselves on various behavioral metrics.  However, after the initial novelty of cool output diagrams and charts wears off, the sensor user is basically just left with a pile of data.  What they then do with it is up to them.  And when allowed to choose between staring at a screen of data or spend that time on just about any other media experience, the average person is going to more often than not choose the latter.

To successfully compete for our attention, sensor data interfaces should take cues from those more alluring alternatives.  As I’ve previously discussed and as the Wired article notes in passing, games may provide clues for how to keep users engaged with sensor data long enough to do something with it.  For instance, gamification tactics – including the now over-hyped mechanics of points and badges, as well as the less commonly promoted elements of narrative structure and role-based teams – can attach positive and negative virtual consequences to certain behavioral outcomes.  In doing so, it can increase the intrinsic motivation of the user to process and act upon sensor feedback data.  That is, users may be enticed to process real data and adjust real behaviors in order to pursue virtual goals.  Further, these virtual consequences can even operate with greater regularity and at higher time resolutions than real world consequences, factors which any reinforcement theorist will tell you are key for conditioning changes in behavior.

The point is that context matters.  Just as sensors augment our perceptual capacities through feedback data, game elements may enhance our motivation to actually make use of that data.  By contextualizing it in a manner that is motivationally relevant – that is, if given a more engaging user interface and inputted into a system  with virtual significance to the user – such feedback data may have a greater chance of steering user behaviors.  Indeed, there is a growing consensus that such small, contextual “nudges” can lead to “disproportionately huge effects.”

Summary

For all three trends discussed above – crowdsourcing through simulation, physiological computing, and ubiquitous sensor technology –  performance feedback plays a key role.  Moreover, in all three cases certain aspects of gaming – the calibration of user intuitions and skills, the pursuit of optimal experiences, and the motivational pull inherent to gameplay – are not only relevant, but may actually drive future development and uptake.  In so doing, gaming may mainstream cybernetic systems designed to “amplify” user creativity, enjoyment, and motivation.  Though this may not lead to walking, talking cyborgs, it is possible we’ll find ourselves with some powerful new tools for discovery and intervention.  At the very least, it should result in some new and fun play experiences.