Quality in Games

How do we define quality in games, and what does it actually mean in practice? In this article I will discuss which definition of quality I prefer, and begin to explore how I apply that definition to games.

“Quality is value to some person.”[1]

This is a quote from Gerald Weinberg that has been discussed to great length in the software testing community [2].

Gerald Weinberg states that quality is relative, and it does not exist in a non-human vacuum. Every statement about quality is a statement about some person(s). More quality for one person may mean less for another. Whose opinion on quality do you take into account when making decisions? In short, the definition of "quality" is always political and emotional, because it always involves a series of decisions about whose opinions count, and how much they count relative to one another. [1]

How does this definition of quality apply to games?

First of all I think about whose opinions about quality that count. Let’s start with the players, whose opinion should matter since they drive revenue. Perhaps I have a specific target group of players whose opinion matters more? If my game is aimed at casual players, perhaps I will not take the hardcore raiders’ opinions into account, or at least not weigh them equally. So my casual player base has some opinions on what quality means for them, and I want to make sure that they perceive the game as having high quality. How do I understand what quality means to my casual players?

I could study similar casual games and see what has been successful and what has been less successful. Try to pinpoint what factors drive value for these players through historical data.

I could also implement different A/B tests [3] and try out different options to see what is more or less valuable to the players.

Gather feedback from the community about different features, bugs, and future potential updates could also be an option, although people sometimes do not know what they want [4].

But apart from the players, there are also other stakeholders whose opinions about quality count. Shareholders, owners, and managers may have conflicting opinions with regards to monetization in games for example. A player may see the most value in getting everything for free, while other stakeholders want to generate as much revenue as possible. A badly designed monetization system that does not generate much revenue may be perceived as having bad quality by both players and other stakeholders, but it might also be that players just don’t use it and it does not affect the value they get out of the game negatively. Since getting everything for free is not feasible, a perfect monetization system may have minimal impact on the player while generating revenue and allowing for further creation of value to the player, in form of new content and features.

To understand what gives these types stakeholders value requires a different approach. Current and future revenue, brand value, current and future development costs, and so on. This is a completely different view of value than what the players have.

It makes sense to make a prioritized list of whose opinions count and their relative order to each other, and then try to make some weighted calculations to maximize the value gain.

My assumption is that something that gives players value often gives other stakeholders value as well, but there are certainly situations where interests collide.

But it is very tricky. Something that is valuable to a game tester may seem pointless to a player or a project manager, but the end result could still be a more bug-free game that also brings value to players and other stakeholders, even though these stakeholders didn’t see the value up front.  

A simple example of a prioritized list of opinions may look something like this:

You could have such a prioritized list either for specific features or decisions, or you could have a general list that you use for all types of decisions. However it might be that different peoples opinions matter more depending on the decision being taken.

It is by no means trivial to try to maximize the value of the stakeholders who matter. But it is something you need to think about when you are making a high quality, high value game.

This is an interesting topic, and something that I will think more about and explore further.

References

[1] Agile and Definition of Quality

http://secretsofconsulting.blogspot.se/2012/09/agile-and-definition-of-quality.html

[2] Quality is value to some person at some time

http://www.shino.de/2010/07/22/quality-is-value-to-some-person-at-some-time/

[3]A/B Testing

http://en.wikipedia.org/wiki/A/B_testing

[4] Myth #21: People can tell you what they want

http://uxmyths.com/post/746610684/myth-21-people-can-tell-you-what-they-want