Designing Need-based AI for Virtual Gorillas

Now that I'm freelance, I get quite a variety of projects to work on. One of the most interesting involves updating the exhibit at Zoo Atlanta, in Georgia. Longtime readers of The Designer's Notebook might remember that I wrote about this project in "The VR Gorilla/Rhino Test" a couple of years ago. When you use the exhibit, you put on a VR headset and experience the zoo's gorilla enclosure as if you were a gorilla yourself. Now the zoo's management has asked me to help them port the exhibit over to new hardware and incorporate new AI for the virtual gorillas.

At the moment, the gorillas don't do much; walking around and exhibiting dominance behavior is about the extent of it. The dominance behavior occurs when a low-status gorilla wanders into a higher-status gorilla's personal space, and it consists of an escalating series of aggressive displays until the low-status gorilla is scared off. It's accurate as far as it goes, but we'd like to extend the behavior model to involve a variety of actions: eating, drinking, sleeping, playing. This will involve creating a model of "gorilla needs" which are fulfilled by these activities in a reasonably realistic manner. That got me thinking about need mechanisms, and they're the subject of this month's column.

A need mechanism consists of several elements. The main one is a variable which describes how much of a given needed object or substance the organism has at the moment. Various events in the world, or activities on the part of the organism, can cause this variable to go up and down. A very simple example is the remaining ammunition in a first-person shooter. Firing your weapon consumes ammunition and lowers the amount remaining; picking up clips off the floor raises it again.

In designing an autonomous organism, such as a virtual gorilla or a 'bot in a first-person shooter, we want them to act on these needs, to do something to satisfy its need when the variable gets low enough. This requires establishing a threshold point at which the organism will start to exhibit the behavior - getting something to eat (for a hungry gorilla) or going to look for ammunition (for a space marine). The behavior is invoked whenever the need drops below the trigger threshold.

However, this creates a problem. Suppose you establish a threshold for remaining ammunition of 10%. Below that threshold our space marine will go look for some ammo; above it he'll continue exploring. What will happen is that as soon as his ammunition drops below it, he'll go pick up a clip, but once he's got that clip, he'll go back to exploring again, even if there is a second clip right there. After he's fired a few bullets the count will drop and he'll start looking around for another clip. He'll be in a very tight behavioral loop, always using clips and picking up new ones one at a time. Similarly, a hungry gorilla would eat one bite, wander off for a few minutes, then come back and eat one more bite, and so on. That's not the way gorillas, or space marines, behave.

What's needed is a second threshold—higher than the first—that tells when to stop the fulfillment behavior. We want the gorilla to continue to eat until she is sated, not just until she is no longer hungry. Similarly we want the space marine to go on picking up ammunition until he's sure he's got enough to last him a while. This mechanism with the two thresholds, one to trigger the behavior and one to inhibit it, occurs quite often in the natural world and in other kinds of devices as well. It's called hysteresis, and it's the reason that the furnace in your house doesn't start and stop every 30 seconds. When you set the thermostat at 68 degrees, the furnace comes on at 68, but it actually goes off at 72. The thermostat doesn't display the inhibitory threshold, but it's built into the machinery.

But now we have another problem. Suppose our space marine has picked up all the available clips, or our gorilla has eaten all the available food. She's above her hunger threshold, but not yet up to her satiety threshold. With the current mechanism, she'll sit there forever, waiting for more food to come along. Our space marine will continue to search for clips endlessly, even though he's got them all. We need to place an artificial limit on the fulfillment behavior if it's no longer successful.

Exactly how this is done depends on how the needed item is distributed, and how smart you want the organism to be. With human beings who know for a fact that there's only so much of the item around, they should stop immediately when it runs out - if you eat all the food in the fridge, you don't hang around the refrigerator hoping it'll magically get some more in it somehow. In the case of the gorilla, though, I would expect her to wait hopefully by the food distribution point for a little while, maybe searching around a bit before giving up. In the case of the space marine, although he may have found all the clips he can see, he also knows that clips tend to be hidden in a variety of places. He shouldn't necessarily stop looking for them as soon as he's picked up all the ones in a room; he should continue to hunt around a little longer, and give up only after he hasn't found any for a while. In each case, we want to set a timer (I'll call it the "persistence timer") every time the fulfillment occurs - the gorilla eats something, or the space marine picks up a clip. It starts to run down while they look for more. If they haven't obtained any by the time the timer runs out, then the behavior is apparently unsuccessful and we interrupt it and return to other things.

However, even this isn't completely straightforward; it depends on the urgency of the need. A gorilla who's eaten enough to not be hungry any more, but is not yet sated, should probably give up and wander off fairly soon. But a gorilla who's still hungry might search for longer, and a starving gorilla might search indefinitely. Depending on the importance of the item, you should set the length of the persistence timer proportionately to the urgency of the need. A space marine who's completely out of ammo should make finding more his first priority, as long as he's not actually under attack.

You can also set additional trigger thresholds to initiate a variety of different behaviors at different levels of urgency. A hungry gorilla will go and eat; a very hungry gorilla might try to steal food away from another one, and a starving gorilla might openly attack another one to get its food. The utmost extreme, at least in humans, is cannibalism.

This brings me to another point: not everybody behaves the same way. Some people would rather die than become cannibals; others have no such compunction. I have long felt that we need to make more of an effort to create unique individuals in computer games. The positions of these trigger thresholds and the length of the persistence timer should be partially randomized for each individual. Some marines are risk-takers, and won't start to look for ammo until they're down to their last bullet. Others are cautious and start looking early. These differences add richness to your game at a trivial cost in code and CPU time, and I definitely intend to add them to the virtual gorilla exhibit. There's a more detailed discussion of this in another early column of mine, "Not Just Another Scary Face."

It's also possible to have too much of a good thing. Just as there are thresholds and behaviors for acquiring more of a needed item, we can also implement thresholds and behaviors for getting rid of excess, at the top end of the scale. In role-playing games, for example, there's often a penalty for carrying too much weight. When this occurs, we want our NPC to dump low-value items out of his backpack until the weight is back down to a manageable level. Choosing which items to dump is of course very tricky, but it's the right response in principle.

The amount of ammunition you have in a shooter is normally only modified by two actions: firing reduces it; picking up clips raises it again. Otherwise it doesn't change. In the VR gorilla simulator, we're going to want the gorillas' needs to change automatically over time - gorillas gradually get more and more hungry regardless of what else happens. Similarly, different activities should affect the needs at different rates. Gorillas that are very active should get hungry faster than gorillas that are sedentary. This whole system is of course how The Sims works, quite explicitly and openly to the player. In the case of the virtual gorillas we're not going to try to train them, so it can all be hidden.

The Sims also nicely illustrates another issue: needs interactions. In The Sims, the simulated people have several needs: food, sleep, entertainment, hygiene, the toilet and so on, but they can only do one thing at a time. The needs are all competing for the sim's attention, and they have to get them all fulfilled or they become unhappy - or worse. To manage this they have a queue of things to do, and it's ordered by the urgency of the need. Using the toilet is at the top, entertainment is at the bottom. Because everything in the game takes a long time, often the Sims never get around to having any fun, and get stressed out as a result.

With multiple needs, you don't necessarily have to wait until the inhibition threshold is reached to stop the current behavior, or until the persistence timer runs out. If all the behaviors can interrupt one another and they can all last an indefinite length of time, you can recompute the urgency of each need every few seconds, and choose a new behavior whenever a new need is more urgent than the one you're currently fulfilling. However, that could again lead to odd behaviors - a person might eat three bites, sleep for five minutes, go to the bathroom, sleep for another five minutes, go eat another three bites, and so on. In The Sims each behavior consists of a series of animations that take a certain minimum amount of time, and they don't generally interrupt the current behavior until it's finished. And of course it's all further complicated by the fact that you can give them instructions which override their own instincts.

The virtual gorilla exhibit won't be as complex as The Sims, but we'll probably want a queue of behaviors as well, sorted by urgency, and a certain minimum amount of time that any behavior can be performed. The urgency is a combination of two factors: how far below the trigger threshold the variable is, and a weighting for the type of need itself. For example, playing is intrinsically less important than eating, but not in every possible circumstance. I'm sure as a child you've had the experience of playing for a long time and having so much fun that you didn't notice you were hungry until there was some interruption. For juvenile gorillas we'll give extra weight to the need to play, which will cause it to be higher in the queue than eating until the gorilla gets really hungry.

One of the classic design questions for computer games is whether urgent needs should interfere with performance. Clearly, if you're out of ammo, you can't fire your weapons, so you can't shoot anyone and take his ammunition. If that were the only way to get any, it would create a deadlock. Most shooter games break the deadlock by providing caches of ammunition that you don't have to shoot anybody to get, or letting you use non-firearm weapons to dispatch enemies without needing any ammunition.

With health, however, the situation is different. Obviously somebody who's near death from bullet wounds shouldn't really be able to run around and fight at top speed, but most shooter games simply ignore this. If you create a performance penalty for damage taken in an action game, negative feedback sets in much too quickly and you don't have a chance. In a war game, on the other hand, things move more slowly and you can often compensate for damaged forces with sound strategy or efficient production. Besides, damage is supposed to convey an advantage to the other side; that's the point of causing it.

I don't anticipate having any such problems with the virtual gorillas. For one thing, they don't kill each other - their dominance behaviors, while dramatic, are not life-threatening. The only resource the gorillas can't generate for themselves is food, and we'll make sure it's provided in abundance so that we don't get any deadlocks. I'd like to create a feedback loop in which a well-rested gorilla becomes a playful one; a long-playing gorilla becomes a hungry one; and a well-fed gorilla becomes a sleepy one, so that we'll see a regular round of activities. We'll have to keep a sharp eye out to make sure that it's a stable loop, though. Too much feedback and they start playing, eating and sleeping faster and faster; too little and they slow down and do nothing.

It's going to be a fun project.