How a Game Tester Adds Value

How does a professional game tester add value to a game development studio? Can you measure this value in a good way? I will try to give my view on this subject. I am specifically talking about value added that is part of the tester role. Obviously anyone can add value outside of their specific role – such as bringing coffee to people, or just performing tasks that are not specific to their role.

First an overview of how I think a game tester adds value:

Finding and reporting defects in the game

A game tester’s primary function is generally to find defects or questionable design decisions in the game. This defect is then reported, in a written artifact or verbally, to someone who can make a decision regarding that defect. A game developer, artist, or other role can then, depending on a host of different factors, fix the defect.

With good enough historical data, metrics and understanding of the users, the value of the absence of the defect in the game can then be evaluated. This is a clear value add that the game tester brings to the table.

But there are additional factors to consider before celebrating the value of testers.  A defect requires analysis, and the tester provides the basis for this analysis. If the defect report is of low quality, then suddenly the tester is adding to the analysis cost. This can be mitigated by securing that a proper defect management process is in place, and that the tester knows which information is needed in the defect report.

We also have to consider that every defect submitted that is not fixed has an analysis cost. This means that the only value a tester has created if ten defect reports are submitted and none of them are fixed is some vague information value, but the analysis cost of these ten issues is very real.

So for a tester to add real value the defects reported must provide the right information, and also actually be fixed. This provides a dilemma. Should the tester send defect reports of every single thing they find to the developers for analysis? There probably needs to be some analysis triage before the defects reach the developers to reduce unnecessary analysis cost. This triage could be done by testers, but it could also be done by a stakeholder who can make a decision if the defect should be fixed or not in the first place, before it is analyzed. However there are of course risks introduced when not analyzing all defects that are found.

Finally we cannot forget that the developer that fixes the defect also shares the added value of the absence of the defect in the game. How much each role contributes to the absence of the defect is probably difficult to assess.

Providing support to developers

A part from finding and reporting defects a tester can also provide valuable support to developers in many different ways.

By creating Acceptance Criteria that are testable and understandable to all stakeholders, the tester provides some value, which is hard to measure. Good Acceptance Criteria bring clarity both to what developers should do, and to what stakeholders actually want. A worst-case scenario with a lack of understanding between stakeholders and developers could be very costly indeed. In a perfect world these Acceptance Criteria can also be used directly as test artifacts, which would free up the testers time to perform more valuable activities than creating test cases, since test cases in themselves do not add any value.

A tester can also add value by supporting the developer with expertise to enable better testing. Helping with test techniques and methods to improve the testing performed by the developer. The value of this support is very hard to measure.

Providing a test environment for the developers to use is also something that adds value. To the testers themselves this is a pre-requisite for their work, but you could argue that this is a service they provide to the developers, which adds value. The value can be calculated by evaluating how much time it would take for the developers to set up and maintain a parallel test environment for themselves. When I say test environment it can include a framework for automated testing.

Testers can also set up, maintain and run different automated test suites to support continuous integration, and other automated test runs. Maintaining and running automated tests have no inherent value in itself for a tester, but can be a pre-requisite for allowing the tester to find defects and provide necessary information to stakeholders. However there is a value in enabling continuous integration, or other development practices that require automated tests. The value add of this activity can be measured in the same way as for maintaining a test environment for developers. How much would it cost for them to set it up themselves?

Testability needs to be built into a game. This is something that a tester can drive at early stages of the game development. By providing knowledge and expertise the tester might get developers to build in testability that would otherwise be forgotten. The value of this is quite hard to put a number on.

Provide test information & knowledge to stakeholders

Also something that is part of the core responsibility of a tester’s job. This can for example be done in the form of a test report, an email, verbal communication, a post-it, or something similar. I here separate between information and information – defects are in a sense also test information, but I separated the two for higher granularity.

So a tester performs a test activity, and submits relevant found defects. Usually the tester also creates some kind of a report with regards to this test activity, which is then sent to a stakeholder. Sometimes this report is consolidated with other testers’ reports by a test lead or project manager. The report created by the tester has information value.

What value the report actually has depends on the decisions based on that report. If a stakeholder reads (or does not read) a report, and the report has no impact what so ever on decisions, then the report holds little value. Some value, but very little.

I have personally been in a situation were the project manager stated that he only wanted a list of new defects, and did not care about the test report. Clearly that report held no value, and I recommended to management that we should stop writing that report.

Sometimes managers can feel more secure if they (think they) have control over what is happening, even though they don’t take any decision based on the report. Then the report has some personal value to the individual manager, but very little to the company as a whole.

But even if the report is valuable to a stakeholder, it is hard to put that value into numbers.

Certification / Mandatory testing

Sometimes there are mandatory tests that need to be executed to be able to release a game. When there are tests that are mandatory to run for some reason, then the tester is immediately providing value by just performing the tests. How much value can easily be calculated by how much it would cost to outsource the test execution to a third-party test house.

Game design / Fun Factor / Balancing / Realism input

A tester spends many hours playing a game. Building up experience with not only a specific game, but with general game design principles. It would be a waste not to utilize this experience.

Some parts of the game design are easier to add value in than others, for testers. Balancing and difficulty level is something that a tester could easily evaluate based on the many hours spent playing the game. At least relative balancing and difficulty level between different components.

Depending on the tester’s other competencies, and available material on how something should work, realism could also be evaluated.

A tester could also provide input on fun factor, but this is highly subjective unless you have the right training in how to evaluate “fun”, I suppose.

Finally, based on the tester’s own game design competence, a tester could provide input on the game design and how it works when implemented in the game.

It seems like a waste not to use the fact that a tester spends so many hours playing a game, and not gather feedback from the tester with regards to these types of game design questions.

Obviously all of these things are very hard to measure the value of.

Conclusion

Of course there will always be other ways for a person in a game tester role to add value to a company than the ones I described above, but I think these are the main value contributing factors the game tester role brings to the table.

In this discussion we should not confuse a person with a role. A person probably adds value in many ways – by being a team player, motivated co-worker, nice person, part-time developer, part-time team leader, and so on – but in the role of a game tester, what I have presented in this article is what I see as the major value add to the company.

But how a game tester role adds value is of course not constant, and my view will change over time, as the field changes and as I get new input that makes me reconsider my standpoint.