Design As Questions, Development As Answers

In Scrum workshops, I often ask developers if they have ever compared the game that they last shipped against the original design document.  The usual answer is that, except for the title, much of the design changed during development.

We are all accustomed to the idea that design documentation is a starting place or a way to win over the stakeholders, but  they are poor maps for development.  The problem is that, lacking a better map, we set off on a development journey that is often longer than planned or takes us to places we didn't want to go. 

Studies have shown that excessive documented specifications can create a false sense of certainty and lead teams astray.  A better approach is to is to acknowledge that designs are speculative and need to be proven out. 

Other industries have faced this problem long before us.  Take, for example, the Boeing 777, the largest twin engine plane built, that revolutionized airplane development and production at Boeing (it was the first airliner fully developed on computers).

One of the many innovations in the development of the plane was to create more concurrency between design and manufacturing (see concurrent engineering).  Before, manufacturing had to wait for all the designs to be complete before they started.  In traditional airplane development, designs are done years in advance of metal bending. Often a problem with design wouldn't be detected until lots of metal was bent and subsequently had to be scrapped, costing millions.  This problem should sound familiar to game developers!

To address this classic problem, the 777 program integrated manufacturing closely with design.  Before the 777 there were only two states for a drawing: released and unreleased.  It was either completely done or not at all.  What they did was add states in between.  Each of these states had to correspond to levels of manufacturing release before they could move the next drawing state.  This reduced the amount of uncertainty that existed with a design: catching problems and introducing improvements far earlier.  This maximized performance with design and manufacturing and contributed to the 777 being developed in record time.

For games, big designs up front (BDUFs) have a similar problem.  A design is either implemented or not, and most of the key design decisions are made without all the information.  We often discover, in production, that we've overlooked some key technical issue that won't allow a feature design to be realized as we had hoped.  We waste effort, throwing assets away and rewriting code to "make it work".

Can we do something similar to Boeing?  Yes.  As discussed earlier, we need to explore our designs and seek out what works and what fails.   Rather than introducing a fixed number of stages in our design, design should introduce questions for core features than then must be answered by development before we move on.

What are some of the questions about core features:

When a design document is written, we don't know the answer to most of these questions, but the true answers that emerge have scuttled many schedules and budgets.  The start of a project is the worst time to try to answer such questions because that is when we know the least!

Take for example, "fully destructible environments".  How many times have we seen this feature in games that don't live up to its promise?  As a developer on a project with this feature goal, my experience was that the surprising amount of production time this feature added ended up killing it and wasted all the development effort we spent in pre-production.  We should have explored the question of production cost well before production started.

Much of development is investing in gaining knowledge about what to build and so making the most of development effort is about gaining the right knowledge, or, asking the right questions.  In the context of Scrum, these questions lead to the priority of items on the backlog, the order of work done in a release and a specific goal for each sprint.