GDC Europe Speaker Spotlight: CCP's David Press On Automated MMO Testing

Author: by Staff

In the latest in a series of interviews with speakers from this August's GDC Europe, David Press, technical director at EVE Online developer CCP Games, discusses the importance of automated testing in MMOs, and why implementing such a system became a "now-or-never decision" for the company. Before joining CCP, Press worked at EA Chicago as a graphics programmer on Def Jam: ICON and an unreleased fighting title. In 2007, he left for Atlanta, Georgia to work at CCP as a graphics programmer, and then moved on to become a systems programmer at the company, and later a technical director. In anticipation of his GDC Europe talk, "Orchestrator: A Postmortem on an Automated MMO Testing Framework," Press explains the various game elements an automated testing system should or should not test, and how CCP implemented such a system for its World of Darkness MMO. What are some of the advantages of automated MMO testing? Most MMOs are very large projects with numerous interacting game systems, and it is simply infeasible to have manual testers constantly testing every nook and cranny of the game. Automated tests allow developers to get wide test coverage before they even check in their changes and they can be run on a changelist-by-changelist basis so that narrowing down bugs is trivial. At what point did CCP realize that automated testing was necessary? What were the driving actors behind the decision? For the World of Darkness project, we started unit testing within the first year of active development, and system testing came in during the next year. We knew that it is nigh impossible to bolt automated testing onto code that wasn't written with it in mind, so it was a now-or-never decision. What sort of blind spots does automated testing tend to have? Are there any specific things that are hard to test with an automated system? Subjective tests are obviously difficult to automate. Whether a texture is rendering or a UI element is correctly aligned isn't worth the effort to test. If you're familiar with the Model-View-Controller pattern, you want to test the interaction between the controller and the model; the view should be verified by your testers. For MMOs specifically, there's definitely a line not worth crossing in terms of how much testing should be automated. For instance, the emergent behavior that we witness in our games when large groups of players interact is often hard to predict and anticipate. We also decided not to write a test for every single part of our systems, but rather test the basic functionality of each system. Especially during development, the details of systems change too often to make it practical to get complete test coverage from your system tests. What is the biggest challenge developers face when writing an automated test for an MMO? There are so many systems that need to be tested in an MMO, so a framework that makes it easy to write these tests is a must or programmers just won't have the time or motivation to do them. Also, because MMOs are distributed systems, you need to be able to write tests that can handle the vagaries of asynchronous communication between multiple, independent processes in a robust, repeatable manner. How will your GDC Europe talk address automated testing in MMOs, and what do you expect people to take away from it? My talk will present a framework for writing and executing automated system tests for the World of Darkness MMO. I will discuss the design decisions that went into the framework and the rationale for them. I will also talk about some of the challenges we still face with our automated tests and how we try to mitigate them. Additional Info GDC Europe will take place August 15-17, 2011 at the Cologne Congress-Centrum Ost, alongside the major gamescom trade show, and will host lectures and panels with other notable speakers, including keynotes from Ultima creator Richard Garriott and Wooga founder Jens Begemann, and more. For more information on GDC Europe, please visit the official GDC Europe website, or subscribe to updates from the new GDC Europe-specific news page via RSS, Twitter, or Facebook. GDC Europe is owned and operated by UBM TechWeb, as is this website.