Quality, Testing & Agile Methodologies

This article sums up why I think Agile methodologies are a key component to attain high quality in complex product development, such as when producing games.

Johan Hoberg, Blogger

December 14, 2016

6 Min Read

My first ten years working with software testing I spent in projects driven according to the waterfall model. But the last 2.5 years I have learned a lot about Agile methodologies [1], and worked in projects adhering to these methodologies to different degrees. In this article I will try to sum up why I think Agile methodologies are a key component to attaining high quality in complex [2] product development.

Complexity

When developing complex products or solving complex problems (as defined by the Cynefin framework [3]), there are many unknowns caused by the very nature of the problems we are trying to solve. This makes it impossible to not only make a lot of detailed plans upfront, but also to set accurate deadlines early in the project. Until we have actually tried to solve the problem, we have no knowledge of all the unknown problems lurking in the shadows. And these unknown problems in turn might obscure even more problems. This means that any detailed upfront design or planning is a fool’s errand, no matter how much experience you have with similar projects, or how smart you are. This also means that there is very little hope of top-down control – as a test lead, development lead, project lead, or line manager, you will have a very hard time to do anything unless you are in the trenches yourself. This means that managers of different kinds must stop asking for different reports and milestone to try to assert top-down control over the development work, because it is a hopeless endeavor. Imagine someone standing at the sidelines trying to assert top-down control over a bull rider. Coaching in real time is possible, but not much else. The Scrum Framework can help you with how this coaching in real time should be handled. What you need to do is to trust the technical experts (and the bull rider) to do their job …

Trust

Modern motivational research shows that what drives people when it comes to solving complex problems is intrinsic motivation [4], as opposed to money, titles or a pat on the back. Self-determination theory [5] states that if you are given autonomy to do your job, you are continuously developing your skills, and you are working with a group of people you get along with (paraphrasing for simplicity here), then this drives intrinsic motivation. So if you are given trust by your managers to autonomously do your job, you have the skills to do it, and you have a team to work with, then we are in quite a good place to solve those complex problems. But if you are given that trust, then this requires you to take ownership …

Ownership

When you take on a complex problem you need to own it. You need to solve it, you need to be transparent about it to everyone else, and you need to make sure that it adds value to the product. If you take it on, you need to see it through. But no one has all the right competence to solve every problem. This means involving the right people. If you have developed a complex piece of code, you are responsible for making sure that it lives up to the standards set by the Definition of Done [6] for the team you are working in. This means, among other things, that you have to test it to the best of your abilities, and if you think that is not good enough, involve an expert in the area. But involving other people does not absolve you from ownership. Just as your managers trust you, you have to trust that other people help you to the best of their abilities, and that they are transparent towards you if they see any additional/remaining risks. In the end, you have to get the right people involved at the right time in the development process to help you in areas, which require additional competence …

Testing throughout the development process

Testing complex products can never be done at the end of a product development life cycle. A complex system must be developed from the start with testing in mind to reach a good level of quality. This means that even the design of the code needs to be created to be testable. Writing unit tests, writing integration tests, involving testers early to review and to test, maybe adapting some kind of Test First methodology [8]. Testing yourself to the extent your abilities allow. Because in the end, testing is also a complex problem to solve, and no matter what you do upfront, circumstances will change and new problems will arise. But that is why you have a team to support you …

Self-Organizing Teams

A self-organizing team is a team that autonomously organizes their work. Self-organization happens within boundaries and against given goals. Teams choose how best to accomplish their work, rather than being directed by others outside the team [6]. This includes looking at what competencies are missing from the team, and making sure that they are able to tackle the complex problems they face. So while there is personal ownership of tasks you take it upon yourself to solve, there is a collective ownership of the product that the team is developing. This means that everyone supports and helps each other to solve all the complex problems, because only together can they cover all the competence needed. But just because you are self-organizing and have autonomy to solve the complex problems, does not mean that you can work on whatever you want …

The Product Backlog

The team needs to work on solving the right complex problems, and these are listed in the product backlog [6], which is handled by the Product Owner [6]. Everything we do should always be prioritized in the product backlog, but how the problems are then solved should be up to the team. The product backlog is what ties complexity, trust, ownership and self-organizing teams together, and should be taken very seriously by all parties. This is how management can give trust to the teams, and how the self-organizing teams can take ownership within the boundaries set by the product backlog. But the Product Backlog is not something you put together at the beginning of the project …

Iterative Development with Continuous Feedback

As with all complex product development you need to start somewhere to unravel all the unknowns. Working iteratively, and continuously updating and refining the backlog with feedback from all stakeholders in a transparent way is necessary to know that you are working on what really provides value to the product. Of course you have to start somewhere, but you should understand that wherever you start, that starting point drowning in a sea of unknown risks and opportunities.

In the end, quality is value to some person [7], and in complex product development that is best achieved by trusting the experts, working in small self-organizing teams, to take the ownership to solve the complex problems that stand in the way of delivering something of value, defined by the backlog, to different stakeholders, and by doing so in an iterative and transparent way.