Practical Game Theory - EV, EV, EV!

One of the most important game theory topics for game designers is the concept of EV, or Expected Value. Widely used in poker and investments, it's also a critical tool for game balance.

Expected Value, or EV, is actually a really simply concept, but many people still struggle with it because of misconceptions about probability.

EV(single event) = %ChanceOfEventHappening x ValueOfEventIfItHappens
EV(multiple events) = sum of EV(single events)

Let's play a game where you flip a coin. If you get heads, I'll give you $1. If you get tails, I'll give you $1.  Pretty good game for you!  You are guaranteed a $1! (Assuming coins landing on their edges are not possible.)  In terms of EV, we have:

EV(heads) = 0.50 x $1 = $0.50
EV(tails) = 0.50 x $1 = $0.50
EV(total) = $0.50 + $0.50 = $1

Your EXPECTED VALUE of this game is $1. That means, on average, you will earn $1 each time we play.

Now let's tweak the game.  I will only pay you on a Heads result. If you flip tails, you get nothing.

EV(heads) = 0.50 x $1 = $0.50
EV(tails) = 0.50 x $0.00 = $0.00
EV(total) = $0.50 + $0.00 = $0.50

On average, you win 50cents each time we play.  Still a pretty good game for you.

Let's tweak the game again.  I will still pay you $1 on a Heads, but if you flip Tails, you have to pay me $1.

EV(heads) = 0.50 x $1 = $0.50
EV(tails) = 0.50 x -$1.00 = -$0.50
EV(total) = $0.50 -$0.50 = $0.00

Note that when you LOSE money, you can just treat it as a negative. Math is easy!

Now your EV is $0.00.  You are going to give back as much as you earn, ON AVERAGE.  Is this game "worth it" to you?  Game theory says no, at least until we start to consider the concept of Utility (a topic for a different post).

Yet another tweak: I will pay you $2 for a Heads, but if you flip Tails, you still only owe me $1.

EV(heads) = 0.50 x $2 = $1.00
EV(tails) = 0.50 x -$1.00 = -$0.50
EV(total) = $1.00 -$0.50 = $0.50

Ok, now we're talking again.  This game is worth $0.50 to you on average.

Let's pause and consider one of the first issues that people get tripped up on with EV.  Should you go out and put all your money on a single coin flip since EV is positive?  Not unless you are really a speculator.  EV tells you that the game is profitable to you, ON AVERAGE.  You want to play this game as many times as you can.  Over time, you will start accumulating a tidy profit.  But you might lose 10 times in a row, so don't go crazy betting on only a couple flips.  This is all very easy to see, as we are just talking about coin flipping.

Now let's make the example a little more challenging with a new game:

Draw a card from a standard poker deck.  If you draw anything but an Ace, I pay you $10.  If you draw an Ace, you pay me $100.  Should you play this game?  Let's figure out the EV:

EV(Ace) = 4/52 x (-$100) = 7.69% x -$100 = -$7.69
EV(non-Ace) = 48/52 x $10 = 92.3% x $10 = $9.23
EV(total) = -$7.69 + $9.23 = $1.54

What does this tell you?  It says the game is profitable! Play ball!  On average, every time you play this game, you'll earn $1.54.

This is kind of where people again start to stumble.  How can you earn $1.54 in a game that only deals in increments of $10 or $100.  No combination of draws will ever leave you with one dollar and 54 cents.  Weird!  Not really.  

When you deal with EV, you are dealing with a probability cloud.  You should not become obssed with details of a single transaction.  Think of it as a bird's eye view of what the math demands.  Doing the EV calc for the above game says that it is profitable.  In game theory speak, any time EV is positive, it's a win.

If you have $100 in your pocket, would you play the above game?  What if the $100 was all you had available to buy food for the next week?  EV is positive, so game theory says the *rational* (aka "smart") move is to play, at least until we start considering Utility and Risk of Ruin (RoR), which actually turn out to be critically important topics.  But this example does highlight the role of human psychology.  Even though math says this game is profitable to play, many people would not play it because they fear losing that $100.  Again, this is traditionally considered "irrational", because not playing the game has an EV of $0, whereas playing has an EV of $1.54.

More Advanced Implementations
A really common use of EV is for calculating the value of various poker actions:

If you are in a poker hand with $500 in the pot, 1 card to come (you are on the turn), and you think you are currently behind the other player but can draw out with 8 outs, you can weigh the value of calling his $250 bet with your last $.   There is tons of literature out there on the web and in print discussing poker EV examples, so I won't reconstruct a standard one here.  But it just involves calculating the EV of losing scenarios (if any card but one of those 8 comes) and the EV of winning scenarios (if any of your 8 "outs" hit), and then adding together.

EV can handle more complex situations than my eariler examples. I can set up a more detailed game such as the following:

Should you play this game? It's really hard to tell on gut feel!  But EV makes it pretty easy. Just set up the individual EVs and then add them all together. If the result is positive, then the game is profitable.

As it turns out, the above game is worth -$3.21.  Here's how:

EV1 = 4/52 x -$100 = -$7.69
EV2 = 1/52 x -$500 = -$9.62
EV3 = 4/52 x $25 = $1.92
EV4 = 2/52 x $75 = $2.88
EV5 = 42/52 x $11.50 = $9.29
EVtot = -$3.21

Game theory says you shouldn't play it, because the EV is negative.

How many of you play D&D?  When I played 1st Edition back in the day, I loved the Treasure Tables in the DMG and Unearthed Arcana.  They were cascading lookup tables: roll first to determine what type of item; roll again to determine what sub-type.  You keep rolling until you hit the bottom and figure out what type of item you found.  For example: +5 Holy Avenger, or a Potion of Haste.

EV gives you the tools to calcualte the "Expected Value" of one roll on the Treasure Tables.  You could figure out the probability of each item, multiply it by the item's GP value, and then add it up.  That would be the average treasure value.  This would be fairly simple, albeit laborious, to do with a spreadsheet.

This takes us into a practical use of the theory for game balancing.

Using EV to Balance Loot Collection in an RPG
Let's say you are designing an RPG.  Naturally, it is filled with chests (or barrels), which are in turn filled with loot of various types and values--gold coins, gems, magic items, etc.

Usually, you want to match the wealth accumulation curve with the training/progression curve so you know how to price things for various stages of the game.  Put another way, you'd like to know how much wealth a player is likely to accumulate, so you can give them interesting purchasing decisions.

If you can estimate the frequency of occurrence--how many chests or barrels (or combats) the player will encounter in a given adventure--then you multiply that by the EV of each chest and you will get their wealth accumulation:

Wealth = #Chests x EVPerChest

Here, EVPerChest is measured in gold pieces (GP) or whatever other measure you wish.

This is only half the story.  The next half is to figure out that EV.  I would choose an EV that makes sense in terms of magnitude (say 50 gp).  This arbitrary decision will set the scaling for your economy.  The next step is to design a chest loot table that results in a 50 gp EV.  
Of course, you can always go the other way: design a loot table with reasonable increments of loot finds (25 gp, 50 gp, 250 gp, whatever) and then calculate the EV.

For example:

TREASURE CHESTS LOOT TABLE
Chance Value
25% 25 gp
25% 50 gp
25% 100 gp
10% gem worth 250 gp
10% gem worth 500 gp
5% magic item worth 1,000 gp on average

The EV for the above chest is 168.75 gp.

If you know the player will hit 25 chests in the dungeon, then their average take will be 4,219 gp.  Price your armor upgrades accordingly!

This method could also be easily adapted to a session-based runner or similar game where you place gems of varying value on the track:

Blue Gems (1 value): 75%
Red Gems (10 value): 25%
EV = .75 x 1 + .25 x 10 = 3.25 per gem instance

So if you know a player will collect 50 gems on average per run, then that means 162.5 units of wealth collection.  Price your upgrades and soft currency accordingly once taking into account how long a run is (time), what your progression options are, and so on.

EV is sort of common sense played out, so apologies if the above is overly simple.

But I find EV and its identical twin Weighted Averages to be one of the most heavily used design tools in my belt. I use it on virtually every game.

For example, unit balancing for strategy games:

I used this exact technique for Age of Empires DS (amongst other things), and it worked out pretty well.  If there's enough interest, I can post an example of this method in more detail.

THE EXPECTED END OF THIS POST
The concept of EV is not earth-shattering, but it is a clear and simple mathematical framework for valuing game objects, especially in situations with probability-driven distributions.