Milestones and Glass Houses: Preventing Your Development Schedule from Shattering, Part 1

It's a pathetic scene all too common to game development. A week's worth of pizza crusts and coke cans litter the office. Artists and junior programmers nod off in their cubicles, exhausted by a stretch of all-nighters. The project leader paces back and forth, annoyed that the game is long past schedule and worried about whether money will come in time to make the next payroll. Throughout the day a dozen pairs of angry eyes bore into the back of the lead programmer, who continues to plod on. Finally, he turns to face the rest of the team and nods wearily. Now begins the upload of the build to the publisher, a several-hour process that will get the promised delivery to the publisher at 3am on the day after it was due. The next day, the project manager telephones a dozen times before getting in touch with the publisher. The news is not good: the game won't run and a new milestone will have to be submitted.

As a game producer and former programmer, I hate having to reject a milestone. I have too many painful memories of living the hand-to-mouth existence of a poor developer, having to beg a publisher for speedy processing of an advance check. So, as penance for all the milestones I've rejected in more recent times, I've written this paper. It looks at the process of developing games funded by publishers through milestone advances, the many reasons why milestones are rejected, and what

Great Expectations

"I want to fund your game," the Big Money Guy tells you after your pitch your game idea. "But only if it can be finished in time for next Christmas. And I want it done for a reasonable price. Oh, and I want it to be a hit."

You flinch at first. Then you summon up your confidence. "No problem!"

But there is a problem. A big problem. You've doomed the project even before it's begun.

It is possible to make a game quickly. It is possible to make a game cheaply. It is possible to make a game that's great. The problem is that you can only achieve two of these goals simultaneously. To make a great game quickly, it will cost a lot of money. You're going to have to pay top dollar for a large and experienced development team, the best hardware and software, and extremely talented project management. To make a great game cheaply, it will take you a long time to find bargains on people and technology. But a cheap game made quickly will likely be a flop because it will be competing against games that had more time and money lavished on them.

Game development is a marriage of technology and creativity, and these two elements add a high degree of uncertainty to the project's schedule and budget for the following reasons:

Still, many businessmen can't understand why it is that experienced developers can't make products that are on-time and on-schedule. After all, they argue, many creative professionals are required to perform while sticking to a schedule. As for technology, the Apollo program had to schedule the invention of new devices and materials, and we still managed to land man on the moon on time. Therefore, businessmen conclude, you ought to be able to do the same with software.

And you know what? They're right. Many developers (myself included) started out as teenage kids in a garage operation with a "let's put on a show" mentality. Our industry is still quite immature, having yet to acquire the discipline found among other professionals. While creating a product that's good, cheap and fast is an extremely difficult task, the difficulties are not excuses for sub-standard work. There are many techniques and practices we can adopt to improve our performance.

Skill at putting together a realistic budget is a good example. If you underbid a project just to win a contract or you put together a budget without factoring in all the details and contingencies, the people funding the project may hold you to your original budget. Somebody's going to wind up putting in a lot of long hours for free. On the other hand, if you bid a project too high or pad the budget, you run a gamut of risks from having the project be canceled to being sued for fraud. Developing games is not a hobby; it's a serious business that needs to be treated as such. Game developers must exercise good business sense.

But let's not let the Big Money Guys off the hook too quickly.

Many businessmen exert tremendous pressure on the development team from the start to be on-time and on-budget. However, deriving a development schedule to meet business needs rather than basing it on the actual time it takes to develop a project leads only to self-deception and disappointment. To add insult to injury, when the Big Money Guys are pinned down once the developer is out of earshot, they will admit that it is much more important to have a game that's of high quality than one that was on-time and on-budget.

The truth is that you need to have someone pushing for the project being on-time and on-budget, and another person being a champion for the game's quality. Both objectives are necessary to achieving an appropriate compromise. You need to be aware of your time and money limitations when you begin a project, and you need to be realistic about what you can accomplish within those limitations.

Sprinting over the Waterfall

Of course, there's always the agile approach, which has gained popularity within the game development community in recent years. Agile project management sees development as an iterative process, in which the project is developed in stages (often called "sprints) of several weeks duration, at which point the project is evaluated at the end of each stage, and development goals for the following sprint are determined. This differs from the more traditional approach, called "waterfall", where the goals for each stage are determined at the beginning at the project, usually before the project is being signed.

The agile approach has several advantages. The development team can more easily respond to changes requirements, such as when the publisher sees that the finished result is not what they had in mind, or when play testing reveals that the game is not as engaging as it should be. The end result, in theory at least, is a high quality game developed in the least possible time and having a satisfied client.

Unfortunately, this process is very difficult to manage if the participants on both sides are not experienced in agile project management. In a large or technically complicated project, it may be very difficult to assess the effort required to get the initial sprints to produce deliverables that are substantial enough to show a good vertical or horizontal snapshot of the overall game and provide enough information to determine what the goals of the next sprint should be. And if the publisher is not skilled at reviewing early prototypes, they may not provide clear direction for what they want as the goals for the next sprint. This can result in the project getting off-track and consume much more time in additional sprints than originally planned.

Some projects employ both models. The waterfall approach is used to develop primary milestones such as proof of concept, first playable, alpha and beta so that the project can adhere to agreed-upon overall schedule. The agile approach is then used for new features or feature changes so that they can be implemented rapidly.

The Best Laid Plans Of Milestones And Men

For a developer to determine what he can accomplish within his time and budget constraints, he must come up with a plan for how he is going to create the game. The end result of that planning should be a road map of specific steps that the developer needs to take to create the game. Such steps become the development milestones, the safety points at which the publisher doles out advance payments to the developer.

The milestone payment method of development offers several advantages to both the developer and publisher:

Unfortunately, some developers with whom I've worked did not take such planning seriously, which was obvious by the very terse technical design documents they created at the start of the project. They apparently thought that, with the difficulty of predicting creative and technological inspiration, with so many details being discovered or changing during development, how can anyone possibly come up with a plan that's worth the paper it's written on? Why not just design as you go?

Such thinking misses the point entirely. As Dwight D. Eisenhower was fond of saying, "In preparing for battle I have always found that plans are useless, but planning is indispensable." Taking time to create a plan has the following benefits:

"But wait!" you say. "If it's so important to carefully craft milestones that will guide the developer and the publisher through the project along the safest path, then why are most milestones written into the contract - before the developer has had proper time, resources and funding to analyze the project?" That is a conundrum, all right, but there are ways around it available to both parties.

The publisher has a strong interest in having the interim steps of the game's development delineated in the contract. Should it ever be necessary to cancel the game, it is easier from a legal viewpoint to determine how far along the project was and what each party's remaining obligations are to each other. However, some publishers create two agreements for each project - one agreement to develop the project up through the design, technical design review, and creation of development milestones, and a separate agreement to develop the project from that point onwards. Thus, the publisher keeps inherently flawed milestones from being put into the contract while minimizing his exposure should he have to cancel the project.

If the publisher won't enter into such a two-stage development agreement, the developer still has some recourse should he discover that his initial milestone schedule is flawed. He can amend the contract with a revised milestone delivery schedule. Both parties must be agreeable to revising the contract, of course. But if the developer does indeed have a compelling argument for changing the milestones, then it would be of mutual interest to make such a revision.

Alpha, Beta, and Gold

Everyone in game development knows what the difference between an Alpha, Beta, and Gold milestone is, right? Wrong! There have been far too many times in my career where I've been assigned to manage a project that is in mid-development, only to discover that the definitions of the Alpha, Beta and Gold milestones where poorly defined in the contract, and when it came time for me to deliver the milestone, the client had a very different (usually much more stringent) idea for what was acceptable than what I had targeted.

Not too long ago, I was assigned to bring to completion a project that another producer had started, and as I was preparing the Alpha delivery, the client told me that they expected the Alpha to be "complete" and "perfect" in every way. Well, to me, that's what the Gold Master is supposed to me, but when I asked the client to explain to me what the different was between Alpha, Beta, and Gold Master was, they couldn't. They just reiterated that the Alpha needed to be "perfect" before paying us for it, and unfortunately, there was nothing in our contract that stated otherwise.

Now, according to Wikipedia, an alpha deliverable is "the first phase to begin software testing… and can be unstable and could cause crashes or data loss." Beta "generally begins when the software is feature complete. Software in the beta phase will generally have many more bugs in it than completed software, as well as speed/performance issues and may still cause crashes or data loss." A Release Candidate "is a beta version with potential to be a final product, which is ready to release unless significant bugs emerge." However, even those definitions leave too much open to interpretation.

To avoid disagreements at the critical time of milestone delivery and payment, the milestone requirements should be defined during in the contract. There are three aspects of each delivery that need to be agreed upon:

Prepare To Submit

Would you put your money into something sight unseen? Well, you might risk a few hundred dollars on a friend's hot stock tip. But what if it were a few hundred thousand dollars? No, of course not. You would want to thoroughly check into the investment first, which is why publishers require developers to submit milestones to them for evaluation.

Now, developers desperately want publishers to pay money for delivery of their milestones. But publishers won't give up the money until after verifying that the milestones are what they are suppose to be. So you would think developers would do everything in their power to make it easy for the publisher to evaluate the milestone as quickly as possible, right?

Actually, you might be surprised how difficult some developers make it to evaluate milestones. Some developers submit milestones that require the installation of a specific version of some obscure third-party software, but they do not go to the trouble of providing copies of the software itself.

One trick for speeding up the payment process employed by many developers I've known is to submit an invoice for the milestone a couple of weeks before the milestone submission itself. The theory is that this will allow a check to be ready by the time the milestone arrives. Of course, they assume that the publisher will immediately approve the milestone upon submission. Unfortunately, that's not always the case, as we will see in the Part 2.