Statistical Analysis to Complement Casual Game Development

Have you ever watched someone else use software you designed or built? It is an invaluable resource for a developer to see an average user interacting for the first time with a program that's been under production for weeks or months. Actions that are second-nature for the programmer are alien and difficult to decipher for a new user. If you're a decent developer you can't overlook the benefits of user testing, but unfortunately you can't look over everyone's shoulder to watch them play your game. However, with just a few post requests sprinkled throughout a casual HTML5 game, you can gain valuable insights into how thousands of users are handling what you've created. For an example, let's take a look at a couple of very simple games I have published.

Every game can have a learning barrier, one concept or task that is likely to cause some players to give up and leave. In desktop or console games these barriers can sometimes be higher, as the user has already shown the level of commitment to download or purchase the game. Some barriers are inherent to the main concept of the game itself; if a user can't overcome the obstacle, they won't be able to overcome future obstacles the game throws at them. It is common sense that the most difficult barrier should not be the first to face the user. If you were faced with the final boss at the beginning of Super Mario, there would be too many challenges without much stake in the outcome of the fight. Now of course for these big-name games, one difficult level is not likely to cause players to turn it off and walk away forever. Unfortunately, in the world of casual and mobile gaming, these barriers can mean the end of the user-experience. It is more essential for casual games than for any other platform to have a smooth difficulty progression, and looking at simple statistics at a macro level can help developers to see a path towards smoothing the difficulty curve.

There are many approaches towards gathering statistics that will be useful in this situation. You could store and analyze every user interaction, each click of the mouse or touch of the screen. This might provide more useful data, but then again, the Library of Babel has all the data you could need, but sifting through it would take too long to be useful. For my games I take the simple approach of recording the initiation of the game, the final score, and the time spent on each game. That only requires two post requests and two tables in a database - not too time consuming. Let's take a look at the data from two of my games, Punt Hooligan and ShockRipple, each with over 10,000 plays. There can be a lot to learn from just a couple data fields.

Comparing Game Starts to Completions:

An obvious starting point would be to analyze what percentage of games are completed. Depending on how you set up your statistic monitoring, this metric could be more or less useful. In Punt Hooligan, the goal is to kick a soccer ball as far as possible. If your kick is not going very far, you can restart in the middle. This will not count as a completion, but it will register a started game. We can then predict that there may be a low percentage of game completions, which does not necessarily indicate a steep difficulty curve. In ShockRipple, however, if a level is restarted it will count as a game completion, and the score submitted will be half the usual. This means that a game start with no completion does indicate a player leaving. 

The results are about what could be expected from the setup of the two games.

Depending on how your game is designed, a low percentage of game completions could indicate a difficulty barrier that is too high. For these games, it does not seem to be the most useful metric. 

Analyzing Time Spent in Game

Another metric available to us is to calculate the time spent on each game. This can be introduced with almost no infrastructure, since you can simply subtract the time from the game start from the score submission time. The time spent in each game will vary, of course, depending on the nature of the game. For Punt Hooligan, a longer time correlates directly with a higher score, so the metric is not very useful. On ShockRipple however, by looking at the average time spent on a level we can see how long users are willing to spend. Let's take a look:

The average time spent on a level of ShockRipple is 32.59 seconds. Unfortunately I was unwise in my setup and did not link each score to its respective amount of time spent... this could improve the usefulness of the metric greatly. Even so, this provides a ballpark for level difficulty: if it takes much longer than this to solve, people might start leaving.

Checking the Score Distribution

In any game, I believe, the reward should be linked to the effort put in. Whether that effort be measured in skill or just brute force, a correlation is necessary. What I look for in my games is a gradual, downward curve. Punt Hooligan is a nice example of this, with a couple anomalies:

(You can view this graph live at https://htmlhigh5.com/game/punt-hooligan)

After around the 2000 mark, the scores gradually slope downward, indicating an appropriate increase in difficulty. There is a sharp uptick around the 1500 mark which I have not yet solved. If anyone has any insights I'd be happy to hear them. Of course there is also the large spike at 0. A score of 0 is achieved when the ball is missed completely. This is a core mechanic of the game, but it would still be nice to see it drop lower. Overall I am content with the difficulty slope of Punt Hooligan.

ShockRipple is a different story, however. In ShockRipple if a level is failed, the submitted score will be half the number of the level. This makes the graph a little more complicated.

(You can view this graph live at https://htmlhigh5.com/game/shockripple)

There is a steep peak at level three. When I noticed it I found the reason for the problem: level 5 is a steep difficulty increase! Clearly I failed to create a smooth difficulty curve here. I wanted a clearer look at level difficulty though, and found confirmation through a little bit of SQL querying. I determined the highest level reached by each user using this query:

select x.max, count(x.max) from (select max(value) as max, user_id from scores where game_id = 10 group by user_id) x group by x.max;

Here are the results: 

I didn't do a very good job labeling my axes, but hopefully this makes some sense. As you can see a very large amount of people made it to level 5 and then were stumped by level 6! The other valleys and peaks are probably a contributing factor to the very low number of people that make it even halfway through the game. (1 person out of 1400 has beaten the game... ouch [no it wasn't me]). This is definitely the most useful data so far. If I were to redesign the game I could use this graph to reorganize the levels, moving the peaks further along and the valleys closer to the beginning.

This is just the tip of the iceberg of what you can analyze and improve with just a few simple data fields! Hopefully this will be helpful to you as you develop anything, from games to productivity software. Let me know in the comments how you track statistics for your software!