Design Testing

QA often uncover issues within features that where inherent in the design. Normally this is a result of undocumented assumptions, edge cases or integration pain points with other systems. These issues are typically found during the functional testing phase after it's went through development and reviews. Can Embedded QA prevent these issues before a programmer commits one line of code? I believe we can and internally we've been doing it via a process we calling Design Testing.

So what is Design Testing? Well, I'll first describe what it isn't. Design Testing is not a mechanism to provide feedback or judgments on the feature at hand. Design Testing is reading a design doc, mentally constructing it in your mind (within the backdrop of your product) and walking through it as a user would.

Holding this construct in your mind, you start to poke and prod the feature and start to be aware of what you expect to happen. You then note these expectations down as they tend to form the basis of default behaviors and you can then check if this compliments or conflicts with the stated design.

So why are we doing this? Preventing Issues is cheaper and far more effective than detecting issues. Additionally, a lot of development time can be saved if these pain points are identified and solved in advance. A benefit of mentally probing the mental construct is that if you write down those tests it can form the initial stages of a testplan. To get designers onboard I tell them that I'll be running these thoughts as functional tests later in the final product anyway so they might as well have a strong answer now since it's easier to amend as a document.

It's worth noting that QA often clamor for any Design documentation they can get their hands on, especially early on before the feature is developed. This is typically met with guarded hesitation from Design as QA have historically used this access to give feedback or get their own ideas across. This can be a source of contention so for Design Testing to work, QA need to focus on factual consequences that will occur by following the feature design to it's logical conclusion.

I've found that this approach gives designers more freedom to own decisions. When QA find a potential conflict via Design Testing, the designer can craft a solution free from the context of a current implementation. This is important as in my experience a the question  "how does the game currently react?" is asked first and if current functionality isn't horrible, then it remains as it. The status quo prevails and represents a lost opportunity for a proper solution to be explored.

We're still refining this form of testing but we've already seen a lot of success from this approach. The biggest challenge is being disciplined enough to not give feedback / opinions. When your next feature is in the design stage, give Design Testing a try and let me know if it worked for you.