Exploring New Content and Player Retention Theory

Alex Engel is an Associate Producer at Disruptor Beam, working on Game of Thrones Ascent. This blog article was originally posted on July 10th, 2013. You may read the original here.

In one of my other lives, I am a writer. Many nights I sit down on my laptop and bang away at the keyboard, writing about video games and whatever else comes to mind. Sometimes I’ll be up late at night writing, annoying my wife with the clacking, when I’ll have a burst of inspiration and start clacking even faster, much to her dismay. I got on this topic today and, since our players like when we go into the details of how we make games, I decided to share it with our players.

The question I was thinking about was, how do you compare the benefits of retaining new players versus creating content for older players? Please note that the rest of the blog post is theory-crafting, and doesn’t necessarily reflect how we (or any other game company) operates.

The answer is surprising difficult. The root of the question lies in how long the game remains enjoyable for our players. I took some time to think about this and came up with a simple equation to figure out how many days of playing an average player will take to exhaust all of the content in the game, at which point they have nothing left to do and will churn out:

((H + C + X) * E) / P = L

Where H are the hours of content produced per week, C is the content already in the game, and X are the hours of player-to-player content engaged in per week. Multiplied by the number of weeks the game has been out (E) and divided by the average playtime per day in hours (P) you can come up with a rough idea of how many average days it will take for content to be exhausted for the average player. Finding those variables, however, is where the difficulty comes in.

To find the amount of content per week, we need to look at a couple of other variables: Bandwidth (D), which are the amount of hours per week the team can dedicate to the game; Bugfixing (B), which are the amount of hours per week the team dedicates to bugfixing per week; Project Hours (J), the number of hours the team dedicates to projects per week (performance, other platforms, metrics, shop, etc.); And how many hours of bandwidth it takes to produce a single hour of game content (K). Put that way, the formula for H becomes:

(D - (B + J)) / K = H

To find the amount of player to player activity an average player engages in per week, the formula is a bit simpler. You take the total number of players engaged in PtP per week (A), multiply it by the average amount of time they spend on PtP per week (B), and divide that by the total number of players that logged in that week (D). You end up with this formula for X:

(A * B) / D = X

So to test this formula,let’s assign some variables to each. I have taken the liberty to assign completely random numbers to this formula, which do not reflect our actual work or time spent in any way.Instead,pretend that they are for an imaginary game called Space Cowboys that I magicked into creation last night.

After being broken out, our final formula looks like this:

((((D - (B + J)) / K) / E) + (((A * B) / D) * E) + C) / P = L

Remember, this imaginary game was released 180 days ago and is called Space Cowboys. We have been tracking our Space Cowboys player numbers, and we can plug them into our formula. With these completely arbitrary numbers we can get a result:

((((180 - (120 + 40)) / 8) * 10) + (((200000 * 5) / 300000) * 10) + 100) / 2 = 91

Yes, I know my math and formulas are messy and not ideal.

So the final result is that, on average, after our arbitrary game has been out for ten weeks, our arbitrary players will exhaust all content after 91 days of gameplay. This assumes an average playtime of 2 hours per day and 5 hours of new content published per week, with ⅔ of our players engaging in PtP play for an average of 5 hours per week, and 300,000 weekly players. This also assumes that we stop making content after 10 weeks and that players stop doing PtP after ten weeks.

But wait, there’s more! We forgot something vitally important: Not all players will play indefinitely until all content is exhausted. Instead, many players will become fatigued and churn out at some point in the game. In fact, for the vast majority of games, a huge percentage churn out after signing up and never play the game at all. So for our imaginary game, let’s set up some imaginary churn rates for an imaginary 1,000,000 players that signed up on Day 0, our launch day.

So what our first formula established was the average time for players to run out of content. What we also have to determine are how many of our imaginary players will even be playing by the time they hit that wall. Using the numbers above, we can expect that at day 90, we will have 250,000 players playing our game. At that time, the average player will have run out of content, meaning that 125,000 of those players will have nothing to do. The other 125,000 will have something new to do, but how much will vary.

Put together, the chart of “Engaged Players,” i.e. the players who have not churned out and still have content to do, begins to look something like this for our numbers:

So by Day 180, we could expect Space Cowboys to have 100,000 remaining players, except since we stopped making content on Week 10, and since they stopped playing PtP, they all exhausted the available content and now we have no one left who wants to play our game. Looking at the graph, we really start to see the effects as early as Day 90, since by then many players will have churned out, and others will have completed content. We could have changed this if we’d either built more content, increased the retention of our players, or acquired new players.

So we need to add something more to the table: The incoming numbers of new players who start out, effectively, at Day 0 every time we acquire them and have them start playing our game. To show this, I added two more player cohorts into the mix, with a cohort of 500,000 players acquired on Launch Day + 15 and another cohort of 250,000 players acquired on Launch day + 30. That changes the mix substantially, because we have the cohort of 500,000 players effectively starting at Day 0 on Day 15 of our initial batch of 1,000,000 players, and another cohort of 250,000 players starting at Day 0 on Day 30 of our initial batch, and day 15 of our second cohort. The end result is this:

 

Still pretty grim. By day 180, even though we increased the total number of players by 75%, we end up with only a tiny fraction still playing the game. So what can we do, as a game developer, to stop that? Well, we could change our retention rate. If we found a way to boost our retention rate by 5% at every step, we would have a boost in players playing our game before they exhaust content:



By days 15 and 30, we would have around 100,000 more players playing Space Cowboys. Not too bad for boosting our retention rate up 5%. However, we still end up churning out players by Day 180 because they ran out of content. If we changed that and doubled our content, pushing our exhaustion date out to 180 days, yet kept our retention rate the same, we would see a greater improvement:

By Day 180, we have almost tripled the number of players still playing. The total number of players, however is still low. This brings up a limitation of my graph, because I only extended it out to Day 180. We’ll see a long tail of players that will continue playing after day 180 which should be substantially longer than the Base or Retention model. This model seems like a slam dunk, but the biggest problem here is that content is expensive. Doubling our content in the game would require many, many hours of work and a much larger team. Our numbers above posited that 8 hours of development work was needed for one hour of content. That means for doubling the existing content in the game - from 100 hours of Space Cowboys to 200 hours of Space Cowboys - we would need 800 hours of development time. That time would go to new art, new engineering work, new design, implementation, QA testing, and regression to make sure we didn’t break any of our existing game content too badly.

I’d like to note here that creating more content isn’t a faucet that you can just turn on and off. Not everyone at Disruptor Beam can equally apply their skills everywhere. I am not too good at creating game systems or coding, and some of our engineers would probably run away screaming if I asked them to take support tickets. Hiring people to do additional work also takes plenty of time. From the hiring process, to interviews, resumes, and other legal work, it can take some time to find a candidate. After that comes the lead time before they can begin, and then they have to be trained before they can dig into the issues. That lead time may vary from a few weeks to a few months, so even doubling team size may not show benefits until (sometimes) several months later.

Anyway, back to Space Cowboys. The final numbers comparing the approaches look like this:

We have to balance increasing the stickiness of a game, meaning the rate at which players are retained, especially during those beginning days, and the rate at which we produce content. At some point, most of your original players have churned out, but you also have the opportunity to reacquire them through further updates, expansions, and changes. Keep in mind that while creating content is very expensive, acquiring customers can also be quite expensive. We have to balance the cost of each with the benefit it provides.

Balancing new players, retention, and new content is part of what separates a successful online game company from an unsuccessful one. Here at Disruptor Beam, we're excited to take on this challenge for our players, so they can keep having fun within Westeros.

As always, comments are welcome!