Postcard From SGS 2005: Riding the Cutting Edge of Distributed Intelligence

Tuesday's keynote at the Serious Games Summit in Washington DC presented a contrast to the first day's, with the animated, hyperkinetic Dr. Dave Warner, MD, Ph.D., who is the Director of the Institute for Interventional Informatics and an expert in sharable situation awareness and real-time feedback on important events. The keynote description noted that Warner is "not a serious game developer but his work on distributed intelligence, cutting edge sensor networks, and virtual reality makes him a shared partner in the quest to change the landscape of learning, healthcare, defense and beyond." His intense, fascinating and amusing talk, which touched on how to better solve real-life humanitarian and communication problems, and had definite relevancy to many of the problems of the serious games market, was enjoyed greatly by all those present.

History Of Warner

Warner's slides and talk essentially took us on a chronological joyride through his career, which he freely admits has been overpowered and has left him stereotyped as "the smart guy," putting him in a number of unlikely situations. His PhD thesis, which he lightheartedly derided as having "really obscure graphics," revolved around concepts that the body is part of the interface of the mind, and finding ways to output and harness this. This evolved into experiments in what Warner called "Grok-it science" - one of the central tenets of his talk was that people should always be practical and relate their ideas to real-life problems, rather than abstracting too far and hoping to solve issues from on high.

In fact, Warner's journey into the use of computer graphics to help analyze human output and be controlled by human input started in the '80s when he commandeered a PDP-11 from the biology department of the college he was studying at, and started working on fractals. As he explained: "When we would ask questions, we could accelerate comprehension", thus gaining insight. This "increased perceptual density" was important when he started medical school, and indicated to his advisor that he wanted to study chaos in the brain - to which his advisor's response was: "Son, we try to get rid of chaos."

Nonetheless, Warner worked on visualizing brain activity, rather than sticking with the normal squiggly lines of brain scanning at that time, using the computer to understand better how brain processes are working and accelerate comprehension. These early experiment were successful to a degree, since different pathological problems had different patterns, and Warner moved on to similar analysis of heartbeats in 3D space, wanting them to act as an "admission accelerant," spotting problems early, but, Warner conceded, the medical community was not ready to adapt to these new processes.

Therefore, a different approach was tried, one that actually has some relevance to the serious game community. Warner tried experimenting with a data glove as a "quantitive human performance assessment tool," particular in regard to tremor-prone patients who had Parkinson's Disease. The data glove was able to algorithmically assess the patient's condition, but unfortunately Warner, in his words, "did something really stupid," and demonstrated that the glove was more accurate than the doctors in diagnosing condition severity, thus moving him on to the next stage of his career.

Objects Floating In Space

Warner then moved onto more health-related "serious game" content, this time carried out in the rehab department for those recovering from serious limb injuries or other movement limiting factors. He criticized the general boredom of working with "stupid plastic balls" for rehab reasons, and ended up giving patients real-time interaction with virtual objects through a data glove, which mean manipulation could take place even before the patient could lift real objects. He also worked with stroke victims who can only move their fingers, getting them to trigger commands through finger movements, at a time (10 years ago or more) when such work was relatively groundbreaking.

The continuation of alternative control methods was especially needed for quadriplegics, who needed non-invasive electrodes to interact with the outside world, and which Warner provided by acting on the electric field of eyes moving around. But probably the most game-like solution came with a young patient who'd had reconstructive arm surgery, and didn't enjoy his muscle exercises. Warner plugged his muscles into a NES controller, and he would then play Super Mario Bros. for about two hours at a time, hastening his recovery. Unfortunately, as Warner noted, those in rehab are a population, not a market - no two cases are the same, so crafting individual solutions for each case ends up becoming extremely expensive.

The Military Complex

After all of this work in health-related solutions, Warner then moved on to another prime "serious games"-related subject, albeit not in the training area, but in the interpretation field. He worked with DARPA to get the data glove control method working for military uses, such as moving a small robot through a building to check its safety, and ended up instrumenting soldiers with low cost pressure sensors, rather than an unwieldy joystick which would leave that person defenseless when attacked - this meant that the sensors were mounted in the glove, and by gripping any arbitrary object and pressing down, control would be possible, an interesting concept for the gaming community. In the days when remote video transmission meant heavy, unwieldy equipment, Warner also innovated in developing a vibration-based belt that would change frequency and power to denote closeness to a wall, meaning that soldiers could navigate the robot remotely without a video feed.

Warner brought similar, intriguing ideas to mine detection, realizing that those sweeping for mines were trying to hear data, and couldn't spatially localize it with regarding to where the end of the device was. His solution was to allow the user to make multiple sweeps, and build a 3D image of the data that could be beamed over a head mounted display to the minesweeper - it wouldn't necessarily show the object, but it would show enough information to make a better decision about what to do with it.

Distributed Intelligence

It's been over the past few years that Warner has been working on a similar theme, and one that has vital importance for both the world's medical and intelligence communities, and also significant impact into the serious game and game communities, too, and it's that of globalized, easily communicable information flow. In other words, how do we hook in medical and situational information so that multiple people and agencies, not all of which may speak the same "language," can understand what's going on? Again, this relates to better managing and visualizing information, the core of Warner's expertise.

The first chance Warner had to demonstrate some of his hypotheses was Strong Angel, an Hawaii-based exercise trying to simulate refugee camp response. Warner quipped: "If the Navy says you want to go to Hawaii, get GPS coordinates first!", showing slides of the desolate Hawaiian landscape that the exercise was based in, but pointed out that coordination in cases like this are all about complicated information collaboration. In this case, Warner and his colleagues were experimenting with remote medical monitoring for geographically remote physicians, not least because Army medics, often deployed to situations like this, may be good at blunt trauma in younger people, but are not necessarily experts in cardiology and congestive heart failure. But, streaming video and audio out to doctors in remote control centers, this would allow remote diagnosis, something demonstrated during the exercise.

In trying to convey overall information to a mass of different agencies, it's clear that each person has a "different logistical footprint," as Warner puts it. This means that using individual terms may confuse, and you may need to abstract the information flow to what Warner described as "grokkable situational awareness," and which he showed as a map with a number of torus-shaped objects sticking out of it. Each torus represents a particular variable such as the amount of food, illness, or danger, and the size and color of the toruses will vary depending on the urgency of the problem. In other words, it presents the situation at a glance in visual terms over multiple locations, and this concept is one that Warner has continued to work through to this day, in both real and virtual situations.

It's particularly in the real world that Warner has concentrated, though, and visiting an UNHCR operation in Africa allowed him to understand and assimilate the situation on the ground much better. Warner paused at this point and exhorted, to all of those getting into serious games as a genre, to "think about and observe the ground truth" before you lapse into what he described as "cubicle-think." Thus he tends to apply much of his learning to real-life scenarios, including work on "Shadowbowl 2003," which enacted real-time monitoring for smoke and dangerous substances around the 2003 Superbowl, and used 'data bazookas' to beam massive amounts of information around, allowing information to be passed to multiple agencies who otherwise don't operate on the same channels.

Gaming Lessons

Finally, Warner also mentioned a couple of fascinating "serious gaming" projects that he had consulted on recently. First, in the realm of computer games, Sandia had asked Warner to help build a thing called a "Quadpod," which was a four-person gaming terminal in which, after looking at data streams while the person plays online games, Sandia asked Warner if he could detect emergent leadership behaviors from a simple physical response. It turns out that he could, making this an excellent quantitative assessment method.

In addition, for the America's Army project, Warner applied some modifications of his torus state ideas to give overseers an idea of the behavior and state of a player while playing America's Army, reading in the game state parameters in real time. Again, this kind of real-time assessment tool could be very helpful in training and learning, making Warner an eminently relevant person to be giving this keynote, and allowing him to end on the key concept behind many of his esoteric methods: "Help me think, don't tell me what to think." This neatly ended a frenetic but rewarding keynote which brought some extremely esoteric ideas into an interesting whole, and enriched the audience while doing so.

______________________________________________________