Dedicated audio testing. It's good. Real good.

We have three QA testers at Volition that are dedicated to audio testing to varying degrees.

There's Brendon, who is in an office in the same hallway as the rest of us and is set up with a decent 5.1 system. He is completely dedicated to audio testing. We have Kelly, who is in an office upstairs with a 7.1 audio setup and who is mostly dedicated to audio testing, but who also tests other systems. And we have Christina, who sits with the other testers in one of our QA tester pits and has a stereo system, and she does some audio testing as well.

I can't stress enough how valuable it has been to have them with us throughout development of Saints Row: The Third. Not only are these three highly trained QA testers actively listening to the game and deliberately trying to specifically break the audio, but they're also available for co-op play, for focus testing as we need it, and to provide some level of opinion about the severity of audio bugs. I'll explain some details about each of these and how it has been helpful.

Having the testers deliberately trying to break the audio has been a huge win for us. Because they did it. They broke the audio. Countless times. And in ways that would be highly unlikely for us to discover, but that the unsuspecting player could trip across while doing the random stuff that there is to do in a Saints Row game. The types of bugs where it's like, start this mission, don't do what the mission asks you to but instead visit a store, then exit the store and notice how the mission music is gone. Stuff like that. So, we were able to fix a lot of exceptions to the rules that could possibly pop up during gameplay. This gives our game a more polished feel than if we had never discovered those exceptions. Because those exceptions undoubtedly will happen to someone.

Co-op is a crazy beast to tame in terms of audio. That's an additional (probably massive) blog post in itself. However, for the sake of testing, it's pretty simple why it's a great help for QA to test audio in co-op: it takes two people. Or one person with two development or test kits. And it takes more time to test than single player. It's just a pain in the butt all around. Having audio QA available to test all of the weird and random stuff that can happen during co-op was a huge advantage. We could also enlist a QA member to play co-op with us so that we could hear how it sounded to be the host, or the client, or both if we wanted to hop between rooms. Brendon also had two kits, so he could test co-op by himself and play us the audio from either the host's or client's perspective.

Focus testing new features was also a huge success. It's not unheard of for a Volition audio designer to do some pretty intense Wwise Wwizardry to get sounds to behave in the ways that we wanted and within the technical limitations that we had. What we don't have, though, is very much time to test it as thoroughly as we'd like. So, we could test it to make sure it works in the normal gameplay situations, and then pass it along to audio QA to test more aggressively, especially using the feature in ways that are outside of expected gameplay.

And finally, having a QA tester available to question about the severity of an audio issue was also a big win. The fact of the matter is that pretty much all audio issues, no matter how trivial to the end user, will seem major to the audio team. I'd liken it to a visible seam in the world. The player may not notice it, but an environmental artist will, and they'll want to fix it regardless of how rarely it would be seen by the player. We need ears outside of the audio team to provide a gamer perspective to our audio bugs. I went into Brendon's office more than once and simply asked, how bad is this, really, in your opinion? If you were just playing this at home and this happened, what would you think? And the great thing about our audio QA testers at Volition is that they are, pretty much to the tee, our target audience/demographic.

So, all said and done, I'm really pleased with how useful dedicated audio testing was on Saints Row: The Third. Our players will appreciate it, too. The bottom line is that having dedicated audio QA on the project means a more polished experience for the player. We will be doing this again in the future.