AN ADVENTURE CALLED DATA ANALYSIS

The idea of this article was based upon a talk with the pitching master Walid O. El Cheikh. I was at his workshop about pitching and he asked about my position – data analyst. “I have never worked with an analyst – I would love to know what exactly you do.” During this workshop, we figured out it would be nice to put this on paper and write an article.

So, what do I do as an analyst?

First of all, it is important to mention that I work for PIXEL FEDERATION – a free2play game developer, on a puzzle game project Diggy’s Adventure. Therefore, of course, part of my job is to play our games!

No, seriously, it is very important to know your domain – I’ll explain later.

ORDINARY WEEK

Usually, I start my day by checking emails and Slack. Then, I have a few kinds of tasks:

1. ANALYSIS/REPORT

Every Monday, the analytic department, game designers and a producer sit at a session talking about what comes next and what numbers would be helpful. The result of these sessions is a list of one or more analytic tasks, which help with the decisions the production team makes.

First, I need to understand what exactly is needed and transform it into a question (f.e. from “Are players more prone to finishing the new tutorial?” get “Is the percentage of players who finished the new tutorial higher than the percentage of finishes in the old tutorial?”). This is what we call the “normal” and the “analytic” language.

How can I answer this question with data?

If the data is already collected, then great (but not win, yet). If it is not and the report is needed ASAP, the analyst has to get as closest as possible with the available data and figure a way out. Otherwise, when it is a long-term report, the analyst should suggest a new event tracking fixing the gap in the data (what brings more work for a programmer who needs to implement it into the code).

Okay, great, data is available. However, not one report is made solely with one table. Usually, 3 and more tables are needed, joined together, filtered and possibly aggregated (compressed into a smaller table by making bins of players with the same features – same platform, region, etc.). This data wrangling can take a lot of time since it is an incredibly big amount of data (millions of rows) and comes with a lot of waiting.

It is a typical day - I finally think I am done, but it is not enough. Data does not give me enough information, something is missing or I made a mistake and here I go, the same process again. And the same waiting (or even longer). Also, calculations are often prolonged because of our tools. Sometimes, it happens to get stuck, overloaded or just does not have a good day – restarts and one has to start again.

Once the prepared dataset is saved, my job is to translate the numbers into comprehensible graphs and understand the information the numbers are holding. Now comes the mentioned domain knowledge. One is not only an analyst but also a bit of a game designer. Example – you look at the data of day 1 retention (how many players come back to the game a day after registration) and during week 12, there is a rise. WOW! We must have done something great!

Wait. This and the previous week (number 11 and 12), marketing brought a lot of new players and therefore, the rise in retention is most likely the result of marketing and not game design.

It is also important to communicate a lot with the people you are making the report for (game designers, producers, etc.). They are the consumers and need to understand it. If they do not understand, they will not use data for their decision. This applies to all the steps of analysis. Plus, it often brings new ideas and questions.

Every analysis is also documented in written form – in “normal” language, containing next steps, recommendations or exact results. If the consumer is OK with the analysis, then it goes to “professional eyes”.

2. GAME ANALYTICS SESSION

Every significant report/analysis is presented at the game analytics session where all the analysts have a look at it and may have some remarks, suggestions, ideas for next steps. Even during the analysis, if I am not sure and I want to know their thoughts/ideas, there is a space to bring it up. More brains have more ideas and they look at the same problem from different angles.

YAY! Analysts and consumer(s) are HAPPY! The decisions are now supported by analysis and I can publish it for everybody to see it.

So, what did I do? Do not expect some sophisticated machine learning models. Basically, averages, medians, percentiles, and exact numbers. But even if it was some model, the hardest and most time-consuming part is to clean and transform your data. However, this is a dream scenario, the path is not so straightforward and there are days when something goes wrong.

3. AD HOC TASKS

This is just an example of all the stuff that comes and needs to be done RIGHT AWAY. You put everything down, roll up your sleeves and start looking for a solution, a cause of a problem, etc. And at the end of a day, you haven’t even started to work on your analysis.

4. EVERYTHING ELSE

Even though an analyst may seem like a machine, he/she is not - one has ups and downs, good and bad days and needs a break or work a bit on oneself. In PIXEL, learning is encouraged and provided by: 1on1s - every week on Wednesday, me and my lead sit down to talk about the progress of my tasks - analyses, reporting as well as my own learning, time for learning - I try to select at least half a day per week for my own development, conference - once in a while, depending on a program, workshops - mostly focused on soft skills, last, but not least, COFFEE - some great ideas were born at the coffee table.

SUMMARY

To sum up what was this article about, here are some key takings:

This may seem like very repetitive work, but it is not. Every time, it is a different question, different problem, different adventure :)