The Importance of Playtests 

By Vera Rabkina, Product Manager at Gismart 

Playtests are something many people tend to talk about but give up on far too quickly. There are many reasons why playtests are resisted. These reasons often derive from an emotional connection with the product, making it hard to take criticism, as well as the issue of creating a necessity to reiterate them too often, taking too much time to get valuable product insight.

However, I think the main reason for this is that user testing is challenging to scale up. Each user interview is unique, so it’s hard to compare results and implement strict guidelines. In this article, we will talk about why playtests are critical for product success, and share a Playtest framework we have developed at Gismart, which has demonstrated  excellent results and helped us bring our products to a new level using minimal resources.

Why are playtests so important?

Before we begin let’s take a moment to figure out what we mean by “playtests”. Playtests are a set of user testing interviews when a person who fits the appropriate target audience, is asked to perform a set of tasks pertaining to a product. The interviewer tracks their activities to figure out product flaws as well as to better understand their audience, even taking on board ideas for potential product development. In terms of ‘when best to playtest’, we believe they are useful at almost any stage of product development.

The more you test, the better your understanding of the user experience is, so the better your product fits the user’s needs. As a result, playtests help you to fulfill three main goals:

1. Obtain information early (and make more informed decisions using it)

2. Study various user patterns (since different users act differently, and the development team may have similar backgrounds as well as user patterns)

3. Re-work things less (re-iteration on ideas at prototype staging is much easier to achieve than at the development stage. Better understanding leads to more consistent results)

How to scale-up playtests?

Playtests have become routine in building new games and modules at Gismart. The problem we faced previously is that one person is not enough to conduct multiple playtests and a few people can’t consolidate all of their feedback. To combat this and scale-up our playests, we came up with the following framework.

Step 1. Make a list of the possible hypothesis you want to test before you start. 

You need to go through the app and create a foundation of what you want to test. In a majority of cases, this is something you don’t like about your app or something you are not sure about. However, you also shouldn’t shy away from testing something you assume your users will react to positively. Write these down and assign each to a product screen where you can test your theory.

Step 2. Create a scenario

It's time to create a bunch of real-life scenarios that contain hypotheses from your list. Here, organizing by screen will help you. A scenario should be a short task, 1 or 2 sentences long, which you will ask each interviewee to perform. At the same time, you should have an expected workflow that you can map each hypothesis too.

Step 3. Interview and fill in the form 

Now you are ready for a playtest, and the interesting part begins. Hand out the list of scenarios, and expected workflow to all interviewees who will use the product. They will conduct the play tests with relatively the same input data. After that they fill in the form, marking which hypothesis was proved right or wrong.

It is also highly recommended to leave some space where interviewees can add extra comments. Usually, this is the place from which you can draw a lot of new data. Another good idea is to ask to provide a short bio & demographic description of every interviewee. This will help with creating proper personas and getting more in-depth insight into their provided feedback. As a result, you receive a clear map of data regarding your product. You can trace how user feedback changed as you update the product as well as get statistics on how crucial a specific issue is and whether it's worth investing in fixing it. The best thing about this is that it helps you unify information and involve more people in the activity, creating not just good, but great products.

I hope that this insight into our framework and strategy helps you to better implement, understand and manage playtests. Remember to think of playtests not as tests, but as a journey to better under your customers and product.