The Power of Random Reinforcement

Take a rat and put it in a cage. Put a lever in the cage. The rat will wander around the cage for a while, sniffing the corners, looking around making sure there are not predators in the neighborhood. After a while it will wander by the lever. Sooner or later it will push it.

Hot chunks of cheese, Ratman! A pellet of food dropped down. Wow, look at that, I press it again and another pellet drops down. Jiminy Cricket, rat heaven, here I come!

Now hook up a counter to the lever. Don’t feed the rat every time it presses the lever but only at certain times. Or even better, copy the rat and put it in three separate cages (or if you’re less SciFi minded, start out with three different rats).

In one cage the rat will get a pellet every 50 times it presses the lever.

In another cage the rat will get a pellet at a random interval, at between 1 and 100 presses.

In a third the rat will never get a pellet no matter how much it presses the lever.

What do you think will happen? If you answered that rat C will quit while rat A will presses the lever the required 50 times and gets the pellet you are correct. Rat B, the one with the random intervals, will press the lever as well. But here’s the catch: it will press the lever faster than rat A. Not knowing when you’ll get the pellet is more exciting than knowing that you’ve got to perform your required 50 presses for a surefire Ratilicious Surprise.

Here’s the real kicker though: take away the pellets and rat B will go bananas, clicking like crazy until it keels over.

The Power of Random Reinforcement

I’m not making this up – it’s stolen verbatim from B.F. Skinner, the behaviorist guy who set up all kinds of weird animal (and human) experiments in the mid 20th century. BTW, Skinner was an English major before he turned to psychology and operant conditioning, which goes to tell that what strange creatures breed in the English lit. programs…

But back to our rat.

We’ve got rat conditioning down pat. We know that giving a reward after a defined amount of work is less exciting than giving a reward after a random amount of work. This explains why people keep grinding in MMORPGs and why games like Descent and Pathfinder are so popular: you get to open that treasure chest and maybe, just maybe, get something great.

But wait, there’s an even wider application: whenever you know that you can gain a reward, but aren’t sure if you will actually get it, you will be excited (assuming you care about the reward that is). Take any kind of game where you draw cards as part of an ongoing action. The drawing of the cards will be exciting. They represent the reward (drawing a perfectly matching card/high card) while drawing something you don’t want represent the risk.

Imagine playing Dominion Amazon , or Magic the Gathering Amazon, or anything that relies on that “random sequential” draw, and stacking the deck. That’s right hot-shot, you get to put the cards in any type of order you want. Not only that, you get to put your opponents’ cards in any order you want as well. You’ll still be drawing the cards, but now you’ve removed the element of surprise. Is the game better?

You sure are more likely to win (unless you enjoy losing that is) but is the game better? Or is it now broken?

Those games rely on “random reinforcement” conditioning. You can feel that if you only draw another card you’ll succeed, you’ll get your reward. This is the same type of condoning that makes you buy another booster pack or another lottery ticket or another pull at the one armed bandit (or check your email, but that’s another post altogehter).

Ok, we’ve defined the mechanism, how do we make use of it?

Putting Random into Practice

The most obvious ways are the ones already mentioned: cards and treasure chests. Dice also fall into this category, more strongly if you get to roll several times in a row (you get all sorts of mental fallacies from that, including the ever popular gambler’s fallacy). But there are less obvious ways to use random reinforcements in your designs.

One type of mechanism that generates the same types of tensions is long term strategy. When you set up something and know that there’s a reward at the end but you don’t know if you’ll be able to carry it through then you’re in random reinforcement territory. This can span an entire game (if it’s shorter) or parts of a longer game.

My personal opinion is that war games, at least the medium to heavy ones, generate a lot of random reinforcement. Not because they are random but because there are lots and lots of long term strategies involved. This works across the entire spectrum of war games, from purely tactical, like Advanced Squad Leader Amazon to almost entirely strategical, like World in Flames Amazon and EuroFront Amazon (which, BTW, is still my favorite block war game even though I haven’t had the time to play it for almost two years – oh, well, there’s always retirement). You also get random reinforcement situations from the individual die rolls and local tactical choices, complete with their own gambler’s fallacies: there’s no way that 1 strength unit can hold out any longer. If only I throw another panzer corps at it I’m sure to break through. Which makes you over commit and leaves you weaker at other points and before you know it the enemy’s in Berlin and you’re wondering how that happened. And I really shouldn’t project my own mistakes on an anonymous reader. You’re probably a better strategist than I am.

Well designed random reinforcement works between games as well, especially short games. It’s hard to predict when designing but you see it when you encounter it, that feeling of “it doesn’t take that long, it’s not that late, we could play another round right now”. I’m not saying that it’s always a sense of random reinforcement that generates those feelings, but that the psychological mechanisms, the unfulfilled promises and the tensions generated from them, are similar. You end the game but you’re still in the game.

What I’m saying is that random reinforcement is a temptation tactic, not a surefire way to make the best game. It can backfire completely – just imagine a game of chess where you’d roll a die every time you tried to take a piece and never knew if your grand strategy would succeed or fall to dumb luck. I don’t think that it would make the game interesting for the current types of chess players. Neither would it work if the player didn’t have some sense of how likely they are to win. That’s why one armed bandits come up “almost win” so often – that generates the sense that “I could have made it right there”. And I imagine that the rat C no reward scenario would work pretty badly once players figured out that a certain tactic had zero chance of succeeding, especially if it was a tactic that gave the appearance of being plausible due to theme or similarities to other games (can I get a chorus of “broken, broken” here?).

But if you’re aware of the psychology behind random reinforcements then you can look to your designs and search for places where it can be applied or strengthened.

More psychology articles:

This post previously appeared on Wiltgren.com - Game Design, Writing and Personal Development. New updates every Monday and Friday.