Muddled Mobile Metrics

This article was originally posted on http://f2pgameanalytics.blogspot.com.

Muddled Mobile Metrics

I love numbers! I’ll play puzzle, logic, or strategy games until the cows come home. I studied math in college, and now I’m a data analyst - my job is to look at numbers all day! So it might come as a small surprise that there are metrics out there that annoy me. In the mobile space, we are reporting metrics that communicate very little information about what they are supposed to be measuring.

DAU/MAU

The ratio of daily active users (DAUs) to monthly active users (MAUs) is supposed to quantify the engagement or stickiness of a game. In mobile right now, a good DAU/MAU ratio is over 0.2. If the number of people who have logged in today is over 20% of the number of people who have logged in over the past 30 days, and this is consistently true over an extended period of time, that user base is definitely engaged with the game. However, every day thousands of apps are acquiring new users both organically and through advertising. This is where it gets tricky.

Let’s use an example with my favorite thing: numbers! Say Game X has 10,000 DAUs and 100,000 MAUs consistently for two weeks. This game is underperforming with a DAU/MAU ratio of 0.1. The developers of Game X decide to acquire 20,000 users and the next day they have 30,000 DAUs and 120,000 MAUs. BOOM! DAU/MAU ratio is now 0.25, that game analyst was wrong! This game has incredible engagement!

Wait, what? Nothing in the game changed. No update, no new content, same old balancing and progression. Depending on the retention of the game, they could maintain this higher DAU/MAU ratio for just a couple days or maybe even one to two weeks. During that time, a naïve developer or innocent investor will think their game is great, and a devious developer can talk about their game with strong engagement.

A better measure of engagement is Sessions/DAU. It communicates much of the same information with respect to a user engaging and interacting with a game, and new user behavior is not volatile.

Unnecessary Data Points

When setting up analytics and event tracking, it is important to ask, “What am I trying to learn from this?” For small studios that don’t have a dedicated data analyst, tracking too much or tracking inefficiently is a recipe for disaster. It will discourage anyone from going back in and sifting through all the noise to find any actionable insights.

At a very high level, almost all game developers want to use analytics to increase engagement, retention, and monetization. With these end goals in mind, some events you do not need to track are:

Menu Clicks

There is no way that knowing a player tapped on menu buttons, pause, or mute would yield any insights toward improving retention, engagement, or monetization. One potential counterpoint is tracking when a user visits the store, but I even see this as unnecessary. Your storefront should be easy to navigate to, one click away from any screen, without being in the user’s face. Monetization metrics like Conversion Rate and Average Revenue Per DAU (ARPDAU) from In-App Purchases, will let you know if users can find the store and if they value the currency.

Use of Consumables

As far as consumables go, the only important element is that users buy the power-up. It does not matter if someone buys consumables and hoards them for later use or they buy a few and use them right away, unless you are using predictive analytics to push a promotion when someone runs out.

Viewed, Cancelled, Failed Purchase

Failed purchase is a QA issue and should be fixed before your game goes live in its respective marketplace. Viewed and cancelled purchases will be reflected in weak monetization metrics, but more importantly these do not tell you the user’s intent. Is that user just checking out what items exist in the game? Is that a potential buyer who needs more currency?

Overly Specific Game Mechanics

It’s a safe assumption that if you made a racing game, people will be shifting up. Everyone. The payers and the non-payers. You will not be able to segment those who have executed the event “Shift_Up” and compare them to the few players that immediately bounced after download.

Level Quit or Retry

If there is a problem with the balancing of a certain level or quest, a basic funnel will quickly identify this. All that is necessary to track is each level complete. Quitting a level or retrying a level will only muddle your event tracking.

Rate Game

You can easily obtain this information from any app marketplace. This is another example of a data point that creates clutter.

Why Does Everyone Love K-Factor?

Part of me thinks I should say K-Factor is silly because it only reflects how often users share the game, and right now it is way more important to focus on making a great game with great content and compelling monetization hooks. Once people are consistently making great free-to-play games, users will share those.

Part of me thinks we in the mobile space should be looking ahead to all the different things we want to measure, and once we have mastered the basics, sure we might want to measure the “sharability” of a game.

K-Factor is important to some, but I can't see how it can help developers succeed with free-to-play games. It comes down to what this metric is being compared against or what is it being used for. If you currently use and understand K-Factor, more power to you.

The mobile space as a whole is making great strides towards creating fun experiences with great content. Analytics, game data, and understanding user behavior are an incredible complement to game design and creativity. Don’t be afraid to start using an analytics product, and don’t overwhelm yourself or your team by tracking unnecessary data points. If you can’t prove something with the metrics you are tracking, you should not track those metrics.

If you have any questions or comments about analytics or the metrics above, feel free to reach out to me at [email protected].