How IBM’s Watson May Be Game Design’s Holy Grail

This opinion piece originally ran on industry game design site The Game Prodigy.  Visit for analysis and case studies.

watson

If you hadn’t heard, IBM’s Watson, a machine-learning computer system, recently competed on the classic trivia television show, Jeopardy!.  Using sophisticated machine learning, 4 years of research from the U.S.’s top computer scientists, and 15 terrabytes of RAM, Watson blew the world away when he beat Ken Jennings and Brad Rutter, the two greatest Jeopardy! champions in history.  Answering complicated questions that would baffle Google’s search bar, Watson showed the world that computer science has developed to the point of actually understanding natural language, of taking the complicated sentences and words that you and I understand flawlessly, understanding them, and then combining that with the light speed computing technology to get us answers.

Often nowadays I feel myself thinking, “We’re in the future!” With all of the amazing technology that we have available to us, mobile devices, wireless technology, Google Maps, Facebook, the PlayStation 3 — it all seems like we are in an age where technology has arrived.  What more is there to invent, really?  What technological hurdles have we not overcome?  With everything we can think of being possible, it’s been harder to imagine what isn’t yet possible.

But after watching IBM’s Watson win Jeopardy!, it’s been the first time in a while I’ve felt like I’m living in the past.  For once I feel ancient and prehistoric in perspective because of everything that Watson has enabled me to imagine.  By watching a computer do what was once unthinkable, it has once again opened up the question: “What else will computers be able to do in the future?”

Personally, I became obsessed with the Watson story.  It’s such a theatrical landmark in human history that I can’t help get excited about what the technology will bring.  Sure, Watson may be as large as ten refrigerators now, but in fifteen years, maybe everyone will have a version of Watson 5.0.  And what new capabilities will that allow?  What new kinds of programs, games, and experiences will we be able to create?

The Holy Grail of Game Design

At The Game Prodigy, we analyze case studies of game designs every day for industry designers to use and learn from.  Many of these designs are “solved” problems; designers set out to achieve a certain gameplay experience, and they have succeeded in designing a system that delivers that.  But there are some problems in game design that are really, really hard, problems that have yet to be solved in any capacity.

Watson brings many new possibilities to the table for these unsolved game design mysteries, a new tool for computer scientists and game designers to use in cracking some of the hardest problems in game design.  Among them is an unsolved problem in game design that has existed since the 1970′s: the problem of the virtual dungeon master.

For decades, the classic tabletop game Dungeons and Dragons has entertained millions.  With its unique design of sending adventurers on quests, into caves, over mountains, into castles and villages and more all with just a set of dice, a few sheets of paper, and players’ imaginations, the game has inspired almost every role-playing-game ever made, from board games to the Xbox 360.  Even if you’ve never dared to play the game yourself, odds are you have come into contact with a game that can trace its design back to D&D.

Dice

While there are numerous D&D Guidebooks and Campaign books that can be purchased and followed, many playthroughs are designed not by the designers behind D&D, but by a dungeon master: a human player who plays along with the others in each game, a key innovation that modern game designers have tried over and over to replicate, with little success.

For those not familiar with the design, the dungeon master is a member of the playing group who doesn’t participate in the same way as the other players.  While every other player has a character they act as, an elf, a mage, a knight and so forth, the dungeon master is in charge of creating the game world and story.  By dynamically weaving together a narrative , the background of the characters, enemies in the world, a map, items, and different challenges, the dungeon master dynamically generates an entire world for the players to enjoy that is infinitely interactable.

The role is thus part game designer, part improv performer, played by a human.  Here’s an example dialog  between some players and a dungeon master, leading them into a treacherous cave:

By having the game and story progression controlled by a human, the players can achieve incredibly fidelity of choices, types of options that have yet to be available in a computer game.  For example, in the above dialog, the players lit a torch, which was then worked into the story by being blown out by the dragon.  They could have done anything: cast a spell, yelled into the cave, thrown a rock, and the game would have acted accordingly.  The dungeon master would have been able to come up with believable responses to all of these: waking the dragon, hearing a snore, or enraging him after he was hit with a rock.

These kinds of interactions are found throughout any D&D game.  Players fight enemies and encounter situations and, instead of just using pre-set abilities as in most modern games, they try to come up with creative solutions.  Firing away at a tree stump to get a redwood to fall on a giant, bribing the butler into letting them into the bar, or playing dead as the savage dogs pass by are all interactions that are possible in the game not because the dungeon master or the game prepared them beforehand, but because the players could imagine doing it.  The dungeon master just has to decide whether or not to allow it, and how difficult it will be in numerical form for a dice roll.

The dungeon master is the ultimate in game design, yet has proven almost impossible to pull off because of its two main components.  While computers have excelled at implementing one part of the design, they have failed miserably at creating the other…so far.

The Numerical Value of Charisma

The first half of the dungeon master design is numerical values, a concept that has been easily picked up by computer game designers.  Until Dungeons and Dragons, abstract human concepts such as “Wisdom”, “Charisma”, “Dexterity”, or “Strength” had never been represented in number form.  These terms were known to everyone, used in stories, novels, plays and films throughout history.  These were simple adjectives in human natural language, but were previously impossible for a computer or game system to understand.

By applying numbers to these concepts, it suddenly became possible to interact with them in a rigorously-designed game environment.  By saying that someone has Charisma of 10, it became possible for D&D to objectively tell the player if they were Charismatic enough for a character to successfully seduce an elf-queen, or if the attempt failed.  When a character has a strength of 3, then they aren’t strong enough to slice through the minotaur’s neck; that feat requires strength of 11.

This was the first wide-spread successful attempt to build a bridge between numerical values, the language of computers, and natural language, the communication tool of computers.  And the design took off.  Today, all modern RPG’s borrow generously from concepts originally pioneered by D&D: health points, magic points, health regeneration, speed, power of flight, strength and defense, intelligence.  These are all values that can be found in any RPG of today, from Mass Effect to World of Warcraft, and even Grand Theft Auto and Madden NFL.

But without the second component, the design of a virtual dungeon master still remains elusive.  The worlds of the aforementioned games contain numerical values, but they are still authored by human designers.  To create a virtual dungeon master, another piece is needed.

A Human’s Understanding

The second necessity to the dungeon master design has proven to be more difficult to emulate with a computer.  It is what seems to be the human touch, the capability of situation recognition.

Applying numerical values to many objects in a game world has enormous value: it has created the modern game industry.  Knowing the stats for a character or an enemy allow a computer to govern their interactions without the help of a human dungeon master.  Designers can develop a game and then ship it to retail, knowing that it will respond to the player’s inputs as they intend it to.

However, these situations need to be recognized and programmed by a designer beforehand.  No computer system today can understand a situation well enough to assign numerical values itself.  If designers create a “Sword” object, then they can manually assign it to have a strength of 10 or a durability of 5, as well as define what “strength” and “durability” do.  They manually create the model, the animation, the sound effects.  They can do this because the designers are humans; they have real-world experiences that they can pull from to create the values of the virtual game world.

But this fails to achieve the holy grail of the virtual dungeon master, because no computer can perform these tasks.  Since a human designer needs to be present at the creation of the rules and values of each object in the game world, it isn’t scalable.  Sure, the development team can create hundreds of weapons, but that will take enormous amounts of time.  Even if the designer creates a procedural generation system to govern the values of different items, such as Spore or Minecraft, it will still apply to only a very limited set of interactions and require enormous time investment.

While procedural and sandbox titles allow players to create things that designers never imagined, computer systems are currently no where near the level of sophistication of a human dungeon master, a program that can create the rules and apply values accurately and on the fly.  Minecraft, is a game where many different blocks are created with rules decided by the programmers.  But a virtual dungeon master would be able to create an “Oil” block that would behave as you would expect, all without any input from the team at Mojang or the player.  It would sit in a barrel, ooze, catch on fire, be sticky, and be found deep underground, and the computer would decide that on its own.  These are the kinds of rules that so far have only been able to be created by humans.

Minecraft

Thus, without the second component of situation recognition, most games are forced to provide options that the game developers have planned out and programmed in: they can shoot all the agents in the room, or they can escape through the door.  What if the player wants to pull the fire alarm and use the water sprinklers to their advantage?  While that option would be completely legitimate in the real world, it’s not possible in the game unless the developers thought of it and programmed it in ahead of time.  The computer can’t recognize the situation and respond.

Today’s games hold nowhere near as many possibilities and as much content as the average D&D game, because today’s computer programs can’t possibly understand all of the situations that are presented to them as a human dungeon master can.  In Halo players can jump, run, shoot, duck, and open doors.  But a player can’t fire at a mountain peak in order to create an avalanche unless the designers thought of it beforehand and programmed it in themselves.  The human dungeon master, the designer, still reigns supreme.

Quest for a Virtual Dungeon Master

While computers have provided us with an amazing array of game titles that have grown and improved over the decades since the game industry’s inception, even today no computer program can compare to the creativity and the level of interactivity of a human dungeon master.

But Watson has opened up a new frontier for computers, a frontier that has the ability to understand situations and create environments that respond as a human dungeon master would.  Watson’s successors have the potential to provide the kind of computing that game designers have always dreamed of, a computer that can understand what players are trying to do and play along with them.

Because Watson is a machine learning program, which means that the rules that govern his answers don’t need to be programmed; instead they are learned from repeated exposure.  By intaking terabytes of documents, files, images, and data and bringing about its own conclusions, it can learn to understand questions and situations on its own.  It would know that oil is sticky, that it is black and flamable, and that it is found deep underground.  Thus, if Watson was dungeon master of a Minecraft-like game, it would be able to create oil for the player to play with, all on its own.

Imagine the possibilities.  A game where you could have the level of freedom that you do in real life, where you could escape from the burning building by building your own parachute and jumping out a window, even though the designers of the game never imagined such a solution.  Or a game where you decide that your partner is a double agent and decide to turn him in, an the game generates a plot and story to follow on its own.  Or perhaps a game with an intelligent dialog system, where players can speak to virtual crime suspects just as they would a human, with all their nuance, subtlety, and misleading characteristics, with the dialog generated on the fly.  A plot and a world that is as flexible as the player’s own imagination.  That is the promise of a virtual dungeon master.

Code

By having the capability of machine learning and being able to dynamically generate fun new scenarios, the choices and options for the player are endless.  An ultimate open world where players can think of the craziest ideas and know that the game will allow them, within limits that seem realistic and fun.

This is all, of course, wild speculation.  Only the scientists at IBM have a good idea as to the future of Watson’s capabilities.  But it is safe to say that we are moving in this direction, and that one day we may actually be in an industry where we can use a virtual dungeon master as part of our tool belt.  No doubt it will pose many new design problems along with answers, but it will be a very exciting future indeed.

This opinion piece originally ran on industry game design site The Game Prodigy.  Visit for analysis and case studies.