Game Analytics 101 Part 4: Cohort Analysis and Segmentation

Your generation is different than your parents’. You don’t have the same tastes, same behaviors, or the same expectations about a lot of things. The same logic applies to the groups that pass through your system. The people who started at the beginning are often different from the ones who started yesterday. If you’re looking at everybody lumped into one pool, you’re going to make mistakes.
 

Luckily, cohort analysis is a straightforward tool that will help you prevent those mistakes and better serve your players.
 

What is it?

Cohort analysis is a fancy way of saying you’re looking at the people who started playing your game or app at different times. Each group is a “cohort” and you should consider all of your metrics as things that may look different for different groups.

How do I use it?
 

Group your players by their starting date. Let’s say you launched your game on January 1 and it’s now August. You might think of the players who joined in January as one cohort, and those joining in February as another, etc. Or maybe you slice by quarter or year.
 

Sometimes you pick cohorts based on your content changes or release schedule, or an event or promotion. For example, maybe you did a major expansion or patch in March. You want to look at the players who joined before that and compare them to the players who joined after.
 

Here’s an example

A team of developers and marketers are using analytics to track their new game, Space Paranoids. The developers made a change to the tutorial level on April 15, so the new player experience is very different before and after that date. The team makes a cohort for players joining up to that date, and another for those joining after it. It should look something like this:
 

With the groups defined, the developers want to see if the cohorts have different rates of completion for the tutorial. The early cohort was completing it 65% of the time, but for everyone joining after April 15, the completion rate was 80%. The fix worked.
 

At the same time, the marketing team wants to know the financial impact. Analytics tells them that the churn rate for the first cohort was 34% and for the second 25%. Meanwhile, the conversion rate jumped from 8% to 12%, and the ARPU from $1.56, to $1.78. Most importantly, the average Social Value of the players in Cohort A was $1.50, but $3.05 in Cohort B, telling them that there was a great viral ripple effect from the improvement. Management is happy, and the team gets kudos for a better design. Bonuses abound.
 

These are simple cases that can be completed in 5 minutes. For more in depth discussion of best practices in cohort analysis, see our blog post.
 

More complex cases arise when we add in segmentation. Segmentation allows you to further break down by CRM groups, geographic regions, gender, etc. In a sense, start date is just one form of segmentation. Let’s say you want to look at two cohorts, but you think that the Australian players and Canadian ones may have different ARPU. To test the theory, you add geography segments to the cohort analysis. This should generate two lines on an over time graph. You could standardize the start dates to be at the same X-value of the graph, or you could look at them in true time and compare patterns.
 

The last post in this series will explore the basics of modeling and predictive analytics.