Building High Quality Games

In this article I will explore what I believe is a good foundation for building high quality games, and software in general. I will cover a wide array of different topics which have in common that I believe they all contribute to this goal. I will not go into detail on every topic, but will try to provide additional reading material when appropriate. This is not an exhaustive list and there are of course other important factors that I will not cover, but I have added those that are most crucial from my perspective. So to iterate and and be bold and clear:

“I believe that the following components are crucial building blocks which enable the creation of high quality games”

To build high quality games we first have to have an idea of what that actually means. I   believe that Gerald Weinberg said it best:

“Quality is value to some person.”

If you deliver a game that is of high value to your customers, then that game has high quality. A software that continuously crashes usually does not provide its users with high value, thus we could consider it as a low quality product. Always think in terms of value to people. Number of bugs, or any other metric does not matter if the value is high, and the other way around. You can have zero bugs and still have a low quality product that does not provide value to anyone.

Read more: http://secretsofconsulting.blogspot.se/2012/09/agile-and-definition-of-quality.html

We are building complex products, and complex problems are defined as problems where cause and effect can only be deduced in retrospect. The implications of this are huge. A communist five year plan for governing something as complex as a country is pointless, since you can never know in advance how to solve this complexity. The same goes for detailed project plans for research and development. Research and development is complex, and any attempts to plan these projects in detail up front is a fool’s errand. You have to inspect and adapt. Start working, inspect the outcome of that work, and adapt to the new circumstances. If you create a plan that outlines the scope of a development project, and you set a date for when you want this project to be complete, the only factor you can modify to compensate for changing circumstances is quality.

Read more: https://en.wikipedia.org/wiki/Cynefin_framework

Read more: https://www.slideshare.net/JohanHoberg/why-all-deadlines-are-bad-for-quality

You can never micromanage your way through complexity. Not only does solving complex problems require a high level of understanding, it also requires the ability to adapt. A manager disconnected from the everyday work does not have the understanding, and cannot make the right decisions for solving these complex problems. Trust needs to be established between management, and the experts solving the problems, and this needs to come with a mandate to adapt to changing circumstances. If you want to build a high quality complex product, you can state what problems you want solved, but not how the experts are going to solve them, because this is not known in advance.

Read more: https://www.slideshare.net/JohanHoberg/quality-testing-agile-methodologies

But for trust to be established ownership must be taken by the development teams. When you work on a product you own the value of that product. You are responsible for delivering value, and that includes everything. It does not matter what your role is, or what competence you have - if you see something that detracts from value, then it is your responsibility to make sure it gets addressed.

To create this trust-ownership culture you need to build relatively permanent, cross-functional, genuine teams. For a team to be able to own the problem or product they are working on, they need to have the entire skillset required to solve that problem or develop that product. As soon as you begin dividing work between teams and have too many dependencies between teams, you are eroding ownership and creating bottlenecks. It is never your fault something is wrong, it is some other team’s problem. When you see a value detractor you assume another team will pick it up.

You also need to build effective teams that are able to deliver high value and that takes time. If you continuously move people between teams like resources in an excel sheet you will never be able to build a trust-ownership culture, and the teams will never reach their true potential.

Read more: https://en.wikipedia.org/wiki/Tuckman%27s_stages_of_group_development

Read more: http://www.scrumguides.org/docs/scrumguide/v1/scrum-guide-us.pdf

Read more: https://rework.withgoogle.com/guides/understanding-team-effectiveness/steps/identify-dynamics-of-effective-teams/

Informational diversity is important in a team, i.e. you need different background, skillsets, experiences and views of problems. Research has found that informational diversity stirs constructive conflict, or debate, around the task at hand. That is, people deliberate about the best course of action. This is key to solving complex problems and providing high value.

Read more: https://www.gsb.stanford.edu/insights/diversity-work-group-performance

When you are putting together your cross-functional team you need to think about the competences needed to solve the complex problems, not which roles you are suppose to have. Roles are not important, solving the problem is. Everyone in the team should do whatever it takes to solve the problem. Whomever is capable should do what is necessary to reach the goal, and not be limited by artificial constructs such as role descriptions. Some tasks might require an expert programmer, while some require rudimentary programming skills. The second of these tasks could potentially be picked up by someone who does not normally write code. If you have the skills to solve the problem, then you solve the problem. So everyone in the team is a product developer, and some have high competence in programming, while others have high competence in testing, art, UI, UX, design, etc. But everyone should strive after having some rudimentary competence in all areas.

To be able to have a high functioning team, not only does it require proper product feedback from stakeholders, but also performance feedback within the team. Stakeholders need to be involved in what is being produced, and that it is of high value, and the team needs to continuously revisit how they work and adapt to changing circumstances.

There needs to be a social contract in place. Everyone needs to know that all feedback is giving with the intent of developing the most valuable product in the best possible way. To help people grow and develop, not to evaluate performance, criticize or assert power.

Read more: https://www.slideshare.net/JohanHoberg/giving-feedback-scrum

We need highly motivated people to solve complex problems. Modern motivational theory promotes intrinsic motivation which can be nurtured in a number of way:

What we do not want is the more traditional extrinsic motivation in the form of carrots and sticks, rewards and threats. To solve complex problems efficiently motivation needs to come from inside. In short, for this to happen, people need to be surrounded by people they can relate to, they need to feel that they are growing, that they are not micro-managed, and that what they are doing makes a difference.

Read more: https://en.wikipedia.org/wiki/Self-determination_theory

Read more: https://en.wikipedia.org/wiki/Drive

You can never test quality into a product. Testing will never assure quality. What it will do is give you information about the state of the product. You can then use that information in any number of ways. So get rid of the old QA acronym, and the old way of thinking about QA and testers as gatekeepers of quality, and instead adopt Quality Intelligence:

“Quality Intelligence is a set of techniques and tools for the acquisition and transformation of raw data into meaningful and useful information for quality analysis purposes"

Quality is owned by the development teams, and testers are part of those teams, but everyone in the team should feel equal ownership of the quality of the product.

Read more: https://www.slideshare.net/JohanHoberg/qi-not-qa-68042026

Testing is not an activity you tack on at the end of your development process, performed by someone outside of the development team. Testing should be part of the development process from the start. During planning, during design, during development, you should always think about testing and quality. Testing is a continuous activity throughout the development cycle, involving everyone in the development team.

Designing your code for testability will much improve your chances of implementing a successful automated test framework and automated tests, but also help make the manual testing more valuable, with for example opportunities to support this with different tools.

Read more: http://www.scaledagileframework.com/design-for-testability-a-vital-aspect-of-the-system-architect-role-in-safe/

Testing is not an activity exclusively performed by testers. Complex testing problems are solved by testers. Obvious problems can be solved by anyone, and that goes for all fields of product development. Obvious programming problems can be solved by anyone with rudimentary programming skills, and obvious testing problems can be solved by anyone with rudimentary testing skills. And everyone has rudimentary testing skills.

Developers test their own code to the best of their ability, and if the code is complex, then most likely someone with a higher level of test competence supports the developers with additional testing.

And everyone in the team helps regression test the game before a release. The Product Owner, the Scrum Master, the programmers, the artists, the game designers, the data scientists. Everyone.

Read more: https://www.slideshare.net/JohanHoberg/defining-test-competence-66709391

There will always be certain competences or equipment that are not feasible to have within a development team. Localization testing could be one example. Testing which requires cumbersome, expensive equipment could be another. Different sorts of certification testing might require an external party. In general you should always look for ways to keep as much of the testing within the team as possible, and make informed decisions on when to outsource testing to other parts of the company or third parties.

Why not outsource all testing and focus on development within the team? There are many reasons for this, but the absolute most important one is: Ownership. The team owns the problem, and the team should solve the problem. If you outsource testing you are most likely also inadvertently transferring some of that ownership as well.

But then there are tons of other reasons: Feedback loops and communication problems. Producing waste in forms of detailed test cases, unnecessary bug reports and additional requirement specifications. Longer turnaround times. And so on.

Read More: https://www.slideshare.net/JohanHoberg/testing-scrum

In almost every case, producing manual test cases and running them over and over again is a waste. Don’t do it. Always assume you product developers with a high level of test competence will perform exploratory testing (you need to learn and understand this concept), and then in the cases where you see an actual value in creating a test case, do it. But also consider writing automated tests in these cases - if you need to create a manual test case for something, then it is almost always better to create an automated test instead.

This is the only way you will be able to handle all the testing required inside you development team.

Read more: http://www.satisfice.com/blog/archives/1509

You will never have time to run all possible tests. You need to prioritize. What input you choose for this prioritization will differ, but once it is done, you will know what to test to get the most value out of your test activity. At some point you will have enough information to make a product quality risk assessment, and then you provide this information to the team and stakeholders. If someone wants to reduce risk even more, you continue your test activity, and if they do not, then testing stops.

Running all possible tests every test activity is not only impossible, since there is an almost infinite amount of tests for every complex product, but even if it was possible it is a giant waste of resources, that could be better spent producing value.

Also stop thinking that your predefined test cases cover all possible tests. They do not.  

Read more: https://en.wikipedia.org/wiki/Risk-based_testing

This is common knowledge by now, but when you work in a team, sitting in isolation working on your own code for a long time, and then trying to integrate it to the master branch is not a good idea. Integrate often, and have an automated test framework that enables fast feedback. Also make sure that you have someone with test competence continuously looking at the master branch, analyzing risk and performing more complex tests when needed. And never merge anything to the master branch without testing to the extent of your abilities first.

Read more: https://en.wikipedia.org/wiki/Continuous_integration

Once a game goes live, unfortunately your ownership of the product does not go away. Make sure to have a close cooperation between the development team and those involved in operations of the live product. Customer Support is one example, IT operations is another. If you have a data science department looking at incoming numbers, then they are also important.  

Read more: https://en.wikipedia.org/wiki/DevOps

Managers should understand that their role as generals on a battlefield is going away. Coaching, enabling and supporting is what a manager should be doing. Highly skilled, driven individuals, grouped together in self-organizing, genuine teams working on complex problems is what is needed today to develop high quality software efficiently. The last thing they need is someone looking over their shoulder, trying to micromanage.  

Read more: https://www.greenleaf.org/what-is-servant-leadership/

Read more: https://en.wikipedia.org/wiki/Servant_leadership

If you have the above listed building blocks covered, you are in a good place to be able to develop high quality games, or any other piece of software. That is not to say that it is impossible to do otherwise, but it is definitely less efficient. Developing high quality games is not the same thing as manufacturing soda cans - what was defined as best practices last century is no longer the best way to run your software development organization. Everyone will need to adapt, or be replaced.