Part 1: The End of the Dark Ages for QA in game development

Teams that blame QA for not finding certain bugs, live in the dark ages of development, because they don't understand that quality (or the lack of it) is the responsibility of the entire team and that QA is only one of the groups contributing to this effort. 

One of the roles of QA is to lead the road to quality by influencing the whole team to put quality at a higher level and by making sure the right tools and processes are in place to enforce quality throughout those processes. 

To be in a position to do so, you need to be empowered by your studio’s leadership and development culture and have the trust and backing from developers and producers.

In the very early days of game development, typically debugging was a developer’s responsibility. About 15/20 years ago it started to become common practice to bring on one or two testers who would test games from the end-user’s perspective.

With the introduction of gaming consoles, game development began to become more complex. Publishers realized that they needed to invest in a larger pool of testers which then became known as Quality Assurance.

Today, a lot of publishers have large pools of testers whereas many developers still keep a smaller team of highly skilled development testers around that are both technical and game savvy. 

Testing was romanticized as “playing games all day and getting paid for it”. It is often seen as a “foot in the door” job. Unfortunately this has at times given QA a bit of a negative reputation within some areas of the games industry.

Fortunately we’re working towards changing the perception of QA and the current industry trend of focusing on fewer but higher quality games helps with that. 

Changing Hiring Standards might also influence this change of perception. I think, or I hope, that those days of advertisements about “playing games all day and getting paid for it” will be history soon.

I just recently read an Interview with Zappo’s CEO about how bad hiring decisions cost his company nearly $100 Million. It makes no difference how large or small an organization is or if you are hiring an entry-level employee or an executive, in Art, Programming or QA etc. 

Our hiring standards at BioWare are high. Our development disciplines attract top talent to work on genre-defining AAA-titles. In QA we set ourselves the same goal. If you want AAA-QA, you need to hire the right people with the right mind-set and the right skill-set throughout the ranks.

Even for our temporary/entry level positions every candidate has to go through a rigorous pre-selection and interview process that have helped us to identify good QA talent. 

I believe that the games industry is undervaluing or even underusing QA in many scenarios. This often leads to frustration among QA professionals who are taking testing seriously, but also feel powerless that their insights into the game are not heard; 

QA traditionally has a high turnover in staff, because it is usually not seen as a career destination within games development. This inevitably leads to loss of knowledge and skill set. High turn-over of staff doesn’t help QA as a department or discipline to mature.   

I also believe that there are still enough folks out there who are under the impression that you can just hire testers off the street, put a controller in their hand and pay them peanuts for "playing games all day" and are then surprised when the output is not as valuable as they expected.

You also often end up with people working on your product who have no personal investment in its success. QA is not easy; but it is rewarding, fun, creative and analytical!

In a good organization, testing is involved in the development process from the outset - from vision & concept over to pre-production, full-production, finaling and post-release support.


Quality Management is done at every point in the game development cycle                                                                                       

At BioWare we involve QA early on to consult with the project teams on the testability of features, identify things that might be hard (or impossible) to test, help develop test approaches in advance of anything being built, raising potential risks and providing feedback from a consumer perspective. 

Quality Assurance at BioWare means working embedded with developers during all phases of the development cycle as an integral part of game development;
Mind you, being part of the scrum team doesn’t mean our QA team is being managed or tasked by the scrum team. This happens at the QA lead level.

Over the last 13 years we have developed a system that supports development in a very efficient way. We gained the trust and respect of the development teams by proving our value. But it was not always like that...

Probably up until Baldur’s Gate II, most QA work was reactionary. The core QA team, thought it would help them to better understand how to test the game if they understood how it was made. 

So they asked designers to teach them and they learned how the toolsets worked and used their friendly relationships with developers to learn more about development and to have more visibility into the development process. Neverwinter Nights was probably the first big turning point for BioWare.

The QA team at this point had started to make test levels using the NWN toolset to test content such as animations, art, assets, etc. - they were testing in a controlled environment and the QA team started to earn the development team’s trust and the perception of QA in the studio started to change. 

During Knights of the Old Republic, the QA Lead at that time (who incidentally is still with BioWare), had started to attend the designer and programmer meetings to get information pro-actively and share it with the test-team and it was during KotoR that the QA team was split into two disciplines - Tech (which generally supports programmers) and Design (which at that time was supporting mostly designers and writers).

The reason for the split was primarily for tasking purposes: the QA staff wanted to split up the developer support work amongst each other and went by their preference/talent for one or the other. 

Jade Empire had established the QA department as part of development and for the first time, QA was invited to planning meetings from the start…. and the rest, as they say, is history :)

Now I understand that not every studio has such a long tradition of working closely together with QA. Remember we’ve been doing this for over a decade. Establishing trust and building relationships with development and production and delivering results had an impact as well on something so basic as our company’s core values and our studio culture. 

At BioWare, we are fortunate that our Core Values and our Studio Culture enable and foster team collaboration where every discipline is seen as equal. QA has a voice, QA is embedded, QA is a truly integral part of development.