Postcard From GDC 2005 - Quality of Life Summit: The Business Case for Improved Production Practices

Held in association with the International Game Developers Association, the second day of GDC included a day-long set of lectures and discussions on the problems of 'quality of life', encompassing long work hours and difficult job conditions sometimes faced by video game industry professionals.

After lunch Tuesday, the Summit presented an hour-long lecture from Steve McConnell, Chief Engineer of Construx Software. McConnell's goal was to illustrate the value of improved planning in software development, for development teams and management alike. Counter to intuition, McConnell explained, greater structure means greater morale, as the team members know what to expect. Greater morale means greater productivity. "Will a systematic approach hurt creativity?" McConnell posited. Not necessarily, he explained. It can, if you're dumb and lazy about how you apply it. Otherwise, structure can be of benefit. It is orthogonal to creativity; there is no real connection between the two. As an example of what an organized team can do, McConnell spoke of the standard ninety-day feature freeze before release; under normal circumstances, where code and design are tested in the execution, this time is required to ensure a program will operate as intended. An organized team, who has already planned its development track, can cut the freeze time in half. They know their software will work, and so they can spend an extra forty-five days adding in "cool things" that otherwise would have been cut.

McConnell began by pointing out that different types of software - business systems, Internet applications, desktop applications, games - all demand different practices, depending on the priorities of the market. They differ in constraints on release timing, ease and quality of distribution, the consequence of defects (economic loss, human life, inconvenience), the number of disciplines (art, music, code) involved, and the amount of polish required. Maybe because of this, the different software segments don't cross-pollinate much. This is unfortunate, as although the field has been around for over thirty years, and just about all of the basic tools have long been created, most software developers don't really benefit from the prior work of others. As such, a huge amount of time in all disciplines is spent constantly re-inventing the wheel. McConnell stresses that, as in any field, methodologies can be chosen consciously, rather than unconsciously; further, the different segments of the industry, each having the different focus it does, have much to learn from each other.

Here, McConnell spit out a list of statistics: that the average schedule overrun may be as high as 100%, that over a quarter of all projects are cancelled, with only about the same amount delivered on time. The reasons for these outcomes, he said, are complicated; they come from a litany of real and perceived inefficiencies. On the real end, most projects are run in an inefficient manner and the average developer reads less than one professional book a year, never mind professional journals. On the other hand, management and customer expectations are often unrealistic and unachievable; likewise, some management and customer actions actually undermine the effectiveness of a project.

The average practice in software development, McConnell illustrated with a graph (percent of organizations on the vertical, effectiveness on the horizontal), is close to the worst performance; rather than a clean bell-curve, with a peak in the center, the graph's curve peaks in the left-hand third of the field. This is contrary to common expectation, although many a programmer will likely nod his head at the figure. In response, McConnell positions ten tough questions for software developers (expecting, he mentioned, that few in the audience would be able to answer more than a couple, if any at all):

• How much are you spending on software?

• How do your teams' skills compare to industry averages?

• How do the capabilities of your organization compare to other, similar organizations?

• What percentage of your costs arise from unplanned rework?

• How confident are you that your "buy" decisions should not be "build" decisions?

• What percentage of your projects are on-time and on-budget?

• How confident are you that your current projects will perform to their estimates?

• What percentage of your current projects are likely to be canceled?

• How satisfied (quantitatively) are users of your products?

• How much (quantitatively) has your productivity improved in the last twelve months?

From here, McConnell argued that, in terms of a return on investment, improved software practices are business's "last great frontier". He showed a list of figures: over a twelve-month period, formal code inspections offered a 250% return on investment; over thirty-six months, it rose to 1200%. Formal design inspections went from 350 to 1000%; cost and quality estimation tools resulted in 250 to 1200%, over those periods; long-range technology planning was 100 and 1000%; another four examples resulted in less dramatic through still large returns, especially over the long term. On average, McConnell said, improved software practices have a return of 500%, after you factor in false starts and abandoned projects; the best organizations have a sustained return rate of 900% for years on end. Considering that, in business, a 15-20% return rate is usually viewed as a no-brainer, it seems weird that software practices are no better than they are, on average.

Where the returns come from is mostly in the latter half of development. In an organized situation, the preproduction stage - from assessing requirements to building architecture, to drafting a detailed design - has only about double the results. Once you get into actual production, though - construction and testing and debugging - the difference between a healthy and a pathological project becomes rather more dramatic, mostly as a result of unplanned rework. On a chart of relative effort required from a development team, double the initial investment resulted in a modest slope increase during the construction phase and a strong dip toward the end, when it came to testing and debugging. The "average" line, by contrast, was far steeper during the construction phase, and continued to increase - though less dramatically - during the final steps of development. A following pie chart illustrated that an average project consists of 60% rework, while an expertly-run project consists 80% of planned work and only 20% of rework.

The results of a well-run project are reduced cost, improved quality, improved cycle time, better predictability, and enhanced morale. McConnell further explained that while a direct return on investment comes from better operational efficiency, indirect returns may be greater: predictability, inter-group coordination, cost control, and risk reduction. He then offered some qualifiers: that the size of required investment varies from project to project, as do payback periods; that not all investments are initially possible, and some depend on earlier investments with smaller returns; and that the best starting point depends on the specifics of your given organization.

If returns are so great, then, McConnell asks, why aren't more companies seizing the opportunity? In response, McConnell showed the classic procrastination poster from despair.com: "Hard work often pays off over time, but laziness always pays off now." He referenced the classic fear of killing the goose that's laying the golden egg. Successful small projects cause complacency, leading to unsuccessful large projects. Most organizations can't see how to apply lessons from other industry segments, and so much time is spent fighting current fires anyway that little time or resources remain to prevent future blazes.

McConnell finished the lecture by offering a number of places for companies to begin, and a general suggestion to start with "low-hanging fruit": that lots of proven practices were readily available, although they might have been proved in other industries. The risk of not adopting these practices, he advised, is far greater than of using them.

In the Q&A session that followed, one audience member mused that, working in the Chinese software industry, he had seen many poor results from good practices, while looking abroad to Japan and the United States he had seen just as many good results from what McConnell describes as poor practices. McConnell was familiar with the perception; he wrote it off to just that; as far as he was concerned, the hard data he had from successful cases showed otherwise. Again he brought up the issue of discipline and creativity, and how the two are not mutually exclusive; the most successful artists are often the most organized.

______________________________________________________