Milestones and Glass Houses: Protecting Your Development Schedule from Shattering

It's a pathetic scene all too common to game development. A week's worth of pizza crusts and coke cans litter the office. Artists and junior programmers nod off in their cubicles, exhausted by a stretch of all-nighters. The project leader paces back and forth, annoyed that the game is long past schedule and worried about whether money will come in time to make the next payroll. Throughout the day a dozen pairs of angry eyes bore into the back of the lead programmer, who continues to plod on. Finally, he turns to face the rest of the team and nods wearily. Frantic hands grab the freshly burned CD-ROM, stuff it into an envelope and rush it off to the local Federal Express office just minutes before it closes. The next day, the project manager telephones a dozen times before getting in touch with the publisher. The news is not good: the game won't run and a new milestone will have to be submitted.

As a producer and former programmer, I hate having to reject a milestone. I have too many painful memories of living the hand-to-mouth existence of a poor developer, having to beg a publisher for speedy processing of an advance check. So, as penance for all the milestones I've rejected in more recent times, I've written this paper. It looks at the process of developing games funded by publishers through milestone advances, the many reasons why milestones are rejected, and what, if anything, the developer can do to avoid re-doing work unnecessarily.

Great Expectations

"I want to fund your game," the Big Money Guy tells you after your pitch your game idea. "But only if it can be finished in time for next Christmas. And I want it done for a reasonable price. Oh, and I want it to be a hit."

You flinch at first. Then you summon up your confidence. "No problem!"

But there is a problem. A big problem. You've doomed the project even before it's begun.

It is possible to make a game quickly. It is possible to make a game cheaply. It is possible to make a game that's great. The problem is that you can only achieve two of these goals simultaneously. To make a great game quickly, it will cost a lot of money. You're going to have to pay top dollar for a large and experienced development team, the best hardware and software, and extremely talented project management. To make a great game cheaply, it will take you a long time to find bargains on people and technology. But a cheap game made quickly will likely be a flop because it will be competing against games that had more time and money lavished on them.

Game development is a marriage of technology and creativity, and these two elements add a high degree of uncertainty to the project's schedule and budget for the following reasons:

Still, many businessmen can't understand why it is that experienced developers can't make products that are on-time and on-schedule. After all, they argue, many creative professionals are required to perform while sticking to a schedule. As for technology, the Apollo program had to schedule the invention of new devices and materials, and we still managed to land man on the moon on time. Therefore, businessmen conclude, you ought to be able to do the same with software.

And you know what? They're right. Many developers (myself included) started out as teenage kids in a garage operation with a "let's put on a show" mentality. Our industry is still quite immature, having yet to acquire the discipline found among other professionals. While creating a product that's good, cheap and fast is an extremely difficult task, the difficulties are not be excuses for sub-standard work. There are many techniques and practices we can adopt to improve our performance.

Skill at putting together a realistic budget is a good example. If you underbid a project just to win a contract or you put together a budget without factoring in all the details and contingencies, the people funding the project may hold you to your original budget. Somebody's going to wind up putting in a lot of long hours for free. On the other hand, if you bid a project too high or pad the budget, you run a gamut of risks from having the project be canceled to being sued for fraud. Developing games is not a hobby; it's a serious business that needs to be treated as such. Game developers must exercise good business sense.

But let's not let the Big Money Guys off the hook too quickly.

Many businessmen exert tremendous pressure on the development team from the start to be on-time and on-budget. However, deriving a development schedule to meet business needs rather than basing it on the actual time it takes to develop a project leads only to self-deception and disappointment. To add insult to injury, when the Big Money Guys are pinned down once the developer is out of earshot, they will admit that it is much more important to have a game that's of high quality than one that was on-time and on-budget.

The truth is that you need to have someone pushing for the project being on-time and on-budget, and another person being a champion for the game's quality. Both objectives are necessary to achieving an appropriate compromise. You need to be aware of your time and money limitations when you begin a project, and you need to be realistic about what you can accomplish within those limitations.

The Best Laid Plans Of Milestones And Men

For a developer to determine what he can accomplish within his time and budget constraints, he must come up with a plan for how he is going to create the game. The end result of that planning should be a road map of specific steps that the developer needs to take to create the game. Such steps become the development milestones, the safety points at which the publisher doles out advance payments to the developer.

The milestone payment method of development offers several advantages to both the developer and publisher:

Unfortunately, some developers with whom I've worked did not take such planning seriously, which was obvious by the very terse technical design documents they created at the start of the project. They apparently thought that, with the difficulty of predicting creative and technological inspiration, with so many details being discovered or changing during development, how can anyone possibly come up with a plan that's worth the paper it's written on? Why not just design as you go?

Such thinking misses the point entirely. As Dwight D. Eisenhower was fond of saying, "In preparing for battle I have always found that plans are useless, but planning is indispensable." Taking time to create a plan has the following benefits:

"But wait!" you say. "If it's so important to carefully craft milestones that will guide the developer and the publisher through the project along the safest path, then why are most milestones written into the contract - before the developer has had proper time, resources and funding to analyze the project?" That is a conundrum, all right, but there are ways around it available to both parties.

The publisher has a strong interest in having the interim steps of the game's development delineated in the contract. Should it ever be necessary to cancel the game, it is easier from a legal viewpoint to determine how far along the project was and what each party's remaining obligations are to each other. However, some publishers create two agreements for each project - one agreement to develop the project up through the design, technical design review, and creation of development milestones, and a separate agreement to develop the project from that point onwards. Thus, the publisher keeps inherently flawed milestones from being put into the contract while minimizing his exposure should he have to cancel the project.

If the publisher won't enter into such a two-stage development agreement, the developer still has some recourse should he discover that his initial milestone schedule is flawed. He can amend the contract with a revised milestone delivery schedule. Both parties must be agreeable to revising the contract, of course. But if the developer does indeed have a compelling argument for changing the milestones, then it would be of mutual interest to make such a revision.

Prepare To Submit

Would you put your money into something sight unseen? Well, you might risk a few hundred dollars on a friend's hot stock tip. But what if it were a few hundred thousand dollars? No, of course not. You would want to thoroughly check into the investment first, which is why publishers require developers to submit milestones to them for evaluation.

Now, developers desperately want publishers to pay money for delivery of their milestones. But publishers won't give up the money until after verifying that the milestones are what they are suppose to be. So you would think developers would do everything in their power to make it easy for the publisher to evaluate the milestone as quickly as possible, right?

Actually, you might be surprised how difficult some developers make it to evaluate milestones. Some developers submit milestones that require the installation of a specific version of some obscure third-party software, but they do not go to the trouble of providing copies of the software itself. Some developers send in milestones that have been "zipped" across thirty floppy disks (and there's always a zip error on disk number twenty-nine). Some developers submit milestones on Syquest disks or Exobyte tapes without first checking whether the publisher has the necessary equipment for reading the media. Then there are the developers who put 100 megabyte files on their ftp sites and expect the publisher to make the effort of downloading them.

For any developer who is serious about making it easy for the publisher to pay them on time, I strongly recommend investing a few thousand dollars on a CD-ROM burner early into the project. I also recommend - I'm about to commit sacrilege here - that the developer include an installer with each programming milestone. Most developers don't create an installer until the Alpha or Beta milestone, but an installer can cut hours or days out of the evaluation process - especially if the publisher needs to put the milestone on many different computer systems.

One trick for speeding up the payment process employed by many developers I've known is to submit an invoice for the milestone a couple of weeks before the milestone submission itself. The theory is that this will allow a check to be ready by the time the milestone arrives. Of course, they assume that the publisher will immediately approve the milestone upon submission. Unfortunately, that's not always the case, as we will see in the next section.

No, No, A Thousand Times No

Nothing can throw a project off schedule more than having to go back and re-do work. In milestone-based development, this usually happens when a developer submits a milestone and the publisher rejects it, requiring a re-submittal. Even if the change required to fix the problem is easy to do, it may take hours or days to pull all of the game's elements together again into a single executable. And if development has been proceeding onward while the milestone is being reviewed, it may be nearly impossible to re-create that milestone with that one problems fixed and no new ones introduced.

All that being said, the publisher has many reasons to reject a milestone - some valid and some… well, let's just say they involve a certain degree of situational ethics. Just to make the subject more palatable, I will present some of these reasons (and what can be done to get around them) as my "Top Ten List of Things Overheard Before A Milestone Is Rejected."

There is some mystical law of programming that ensures that developers complete milestones twelve minutes before Federal Express makes its last pick up of the day. In an age when most developers use a local network for developing games, "completed" often means that they burned the work-in-progress onto a CD-ROM for the first time since submitting the last milestone. Now, common sense suggests that the developer should wait a day to verify that the milestone runs properly, but I've lost count of the number of times I've received milestone submissions that failed to even load into my computer. (Oddly enough, they usually come from the developers who send their invoices in advance of the milestone.)

Even when developers are diligent about testing milestones prior to submission, they may discover that the milestone still doesn't work at the publisher's office. Hardware incompatibility is usually the culprit here. Since compatibility issues are often best addressed in the latter phases of development, the ideal solution is for the developer and publisher to set up identical computer systems at each of their offices. These systems should be alike in every way - right down to the installed software - to ensure identical results at both sites.

There are those people who say that once an agreement is signed, you should lock it away in a filing cabinet and not bring it out again unless there is a problem. So what if the milestone doesn't do exactly what the contract says? Well, I am not one of those people. Should a problem ever arise during the game's development, it may be too late deal with it effectively if either party has not been meeting the terms of the agreement along the way - and that includes the milestone definitions. If a milestone submission fails to meet the terms of the agreement, assurances and promises that the deficiencies will be corrected later are not acceptable. From a business and legal standpoint, the only appropriate thing to do is to reject the milestone, or, if the developer makes a compelling argument for doing so, revise the contract.

The developer can circumvent this problem sending the publisher a progress report on the milestone well in advance of submission. One developer with whom I have worked always sent me a report explaining in great detail how each milestone would be implemented in the next submission. Most importantly, the reported listed features that would not be implemented with the milestone delivery. Consequently, any disagreements about what the milestone should accomplish could be resolved before a resubmission was necessary. However, such precautions work only when the people involved really do have approval authority over the game. I've known developers who exerted a lot of energy attempting to persuade me to send them a letter approving changes to an upcoming milestone, even though many other people in my company were involved in the approval process.

Consider a milestone description that says, "Ability to select and complete missions, allowing player to fight anywhere from two to two thousand Ninja armed to the teeth." Does that mean that the player should be able to fight two thousand Ninjas in this milestone or in some future milestone? Sometimes interpreting a milestone is as contentious as interpreting the Bill of Rights. As a result, the developer can submit a milestone fully believing that he has satisfied the requirements, only to have the publisher throw it back to him as incomplete.

Problems involving interpretation of milestones can be prevented by having the people responsible to interpreting the milestone involved in the creation of the milestone descriptions. It is not practical to create milestone descriptions that cover every conceivable detail of a game's creation - especially since such schedules are usually created before serious planning of the game's development can be done. Therefore, people reviewing the milestone must be aware of the intent of the milestone.

As the game progresses, it's important for the producer to keep their finger on the pulse of the project. It generally makes the developer uncomfortable to work for long periods of time without the producer stopping by to look at the project. Also, if the producer isn't around when the milestone is ready to be submitted, it leaves a bigger gap in the approval process.

Regrettably, not everyone who reviews a milestone is guaranteed to have an informed opinion. While the management of many publishers don't even look at works-in-progress between the Design and Alpha milestones, I've encountered Big Wigs at several companies who look into projects at random intervals. Without knowing the details of what lead up to the milestone and what is yet to come, they sometimes demand changes before allowing the milestone to be approved. Usually this sort of surprise participation in the approval process throws everyone involved in a tizzy and can even throw the entire project off schedule.

Publishers need to understand that very large and complex games always have "just one more bug" and could use a "bit more fine-tuning." Thus, there are degrees of meeting specifications and evaluation of milestones must take this kind of imperfection into account. Unfortunately, this is - and often cannot be - done scientifically. It often amounts to who shouts the loudest, but since micromanaging Big Wigs usually exercise a great deal of influence over the release of funds, they often win shouting matches. Very little can be done except to remain calm, be patient, and try to avoid sharp objects.

I have a confession to make. I once worked for a developer who had grossly underbid a project before I joined the team. To make ends meet, I was occasionally required to submit milestones that were not as complete as the contract required. Fortunately for me, the publisher was not technically hip, and with a little smoke and mirrors trickery from the programmer and a little song and dance by myself, I was able to convince the publisher that the milestone represented more progress than we had actually accomplished.

I do not recommend this practice to other developers. Ethical and legal considerations aside, it usually only serves to delay the inevitable (in fact, the developer to whom I just referred went out of business not long after finishing the game). If you can not stick to the schedule the early milestones of a project, chances are you won't be able to catch up at the end.

As for publishers, I advise keeping a technical eye on the project and not allowing an unscrupulous developer to pull the wool over their eyes. There have been cases where a publisher has paid top dollar for a developer to create a state-of-the-art game engine, only to discover that an off-the-shelf third-party engine was substituted instead, with the rest of the cash winding up in the hands of a Ferrari dealer.

Sometimes the developer satisfies every milestone requirement to the letter of the contract, but the publisher still has a nagging suspicion that something is wrong. Perhaps a key feature that was implemented early on has stopped working in the past three milestones. Perhaps some of the more difficult milestones to program are turning out to be the later ones. Perhaps the list of bug fixes, tweaks, polishing and finalizing for Beta is growing uncomfortably large. Whatever the reason, the publisher is losing confidence in the developer's technical ability to complete the game.

While it would be prohibitively expensive and time-consuming to deliver milestones that were final and perfect in every way, a developer does need to ensure that each new milestone submission does indeed demonstrate progress. In an ideal world, a milestone submission should stand securely on its own without an accompaniment of excuses or conditions, eliciting an enthusiastic response from the publisher.

Admittedly, we don't live in an ideal world, but a succession of milestones that provoke only a ho-hum response from people may be a sign that the project is in trouble. If the developer is indeed following the letter of the contract but does not appear to be inclined or capable of correcting the problem, is there any recourse the publisher can take?

Should friendly persuasion fail to work, there is a clause in most contracts that overrides milestone descriptions: the publisher has a right to cancel the project for any reason. Now, if it actually becomes necessary to make such threats then the project may be in great trouble indeed, and I do not recommend any publisher to make such threats lightly. However, I do wish to emphasize that a publisher should never feel that he has to accept substandard work just because the letter of the contract is being met. Sometimes it is necessary to take a firm stand rather than surrender the project to mediocrity.

Consider the publisher who has advanced half of the million dollar budget, and then realizing that the project is less than a third complete, begins to get a sinking feeling that the project is turning into a money pit. In theory, the milestone deliverables and advance payments should have been arranged so that much of the project's risk is overcome before too much money is invested. But once again, we don't live in that ideal world in which the most difficult milestones are always scheduled first, the developer has the financial wherewithal to take on projects with the advance payments weighted at the end, and the publisher has been carefully watching where the money is being spent.

When the publisher finds himself in a situation in which the budget is out of control, the prudent thing to do is to stop all milestone payments. If the developer and the publisher have an honest relationship with each other, they can then get together and assess the financial well-being of the budget. But money has a funny effect on people, and not everyone is open to discussing their finances. Therefore, publishers have been known to find excuses for rejecting milestones, just to see if the developer can make due with less money. Of course, such experiments may only accelerate the demise of a cash-strapped developer.

One of the most important milestones is Alpha. That's when all the game's elements usually come together for the first time, and people finally get a feeling for what it's like to play. In fact, looking at a milestone developed prior to Alpha can be a letdown to the untrained eye. Often when I've shown pre-Alpha milestones to people who are not in product development, their reaction is, "That's terrible! That's not a game!"

"So what," you say? "What matters is what the game plays like when it's finished! Developers don't create sales and marketing materials, you know!"

Actually, developers do creating sales and marketing materials with their milestone submissions, whether they are aware of it or not. Many people outside of development look at milestones: the publisher's sales and marketing staff, the publisher's management team and investors, potential distributors, foreign localization companies, trade show attendees, the press - everyone who has a future interest in the game, since the milestone is usually the best indicator of what the game is going to be like. On occasion, a milestone that was difficult to run, full of crash bugs, and featured half-finished artwork has so underwhelmed preview audiences that the publisher had to reconsider their investment in the game.

As I stated above, milestone submissions should elicit an enthusiastic response from the publisher, but here I am broadening the definition of "publisher" to include people apart from product development. In crafting the milestone schedule at the project's start, thought should be given to including a "wow" feature within each new milestone - if only to continually build everyone's excitement for the game. The developer should complete all features to his own satisfaction before including them in a milestone. He might lose some time if the work needs to be redone, but he will more than make up for it by earning the publisher's confidence.

Another point worth repeating is that milestones should be easy to install and run so that they can be put onto a conference room computer or a salesman's laptop. It is in everyone's interest to include an installer and all necessary third-party software onto each milestone submission and to ensure that crash bugs do not interfere with one's appreciation of the game. When submitting or reviewing a milestone, consider whether it is worthy of showing at a trade show, because there is always a chance that it may need to serve that purpose. Since all games inevitably wind up being developed late, any milestone that includes a trade show demo will likely be developed late, too.

Some publishers have been know to contract two developers work independently on the same game. The publisher's intent is to hedge his bet: once one developer proves himself to be creating the superior version, the contract with the other developer is canceled. This is capitalism at its best if everyone enters the deal knowing what they're getting into, but in cases where the two developers are unaware of the other's involvement, it does not speak well of the publisher's trustworthiness.

Similarly, a developer's game may be competing against another game created by that same developer. In a case of putting too many eggs in one basket, a publisher and developer may be working on more than one project together. When one project is suffering, the other project may be used as leverage for getting the first one back on track. This stratagem is also used with ports and localizations that are being developed concurrently.

Thus, approval on one milestone may be held up because of another game either doing better or more poorly than the one supposedly being reviewed.

Many development contracts specify a limited time period by which a milestone must be reviewed and either accepted or rejected. The intent, of course, is to force the publisher to provide feedback on the milestone within a reasonable time period. But what is reasonable? In some contracts I've seen, the review period is so short that it fails to account for when people are at trade shows or on vacation. In other contracts, it is so long that it just encourages review of the milestone to be put off as a matter of little urgency.

Not that it matters. A publisher can always find reasons to reject a milestone if the review period is close to expiring, and failure to adhere to the limitations of a review period is not generally considered to be a breech of contract. With this issue, the personal relationship between publisher and developer is more binding than the legal obligation.

Prior to submitting a milestone, the developer should contact the publisher and find out if someone will be available to review it right away. If not, it might be worth delaying the milestone submission a few days to polish it a little more. After submitting it, the developer should follow up with a call to get at least the publisher's initial impression of the milestone.

Even large companies get into cash-flow problems; in fact, they can be harder to get money out of than small companies. Besides the red tape that must be gone through in order to cut a check, some corporate behemoths set aside a specific amount of money each month for development expenses. If the money runs out that month, little can be done about it. Some publishers I've dealt with have been up front with me when the cash is running low, but others find excuses to reject milestones just push off the development payment.

Handling Rejection

Many developers deal with milestone rejection quite professionally, seeing their job to be earning the publisher's satisfaction. With other developers, their reaction is sometimes akin to the stages that patients go through when they learn they have a fatal disease: denial, followed by anger, and eventually, acceptance. Actually, I admire developers who defend their position, especially when they make such a compelling argument, they change the publisher's assessment of the milestone. But beware of arguments of "that can't be done." When I was a programmer, that expression meant either "I don't know how to do it" or "I don't want to do it." A good producer and project manager need to know the difference between the two. In the former situation, they need to have the technical sophistication to discuss with the programmers why something can't be done.

It is damaging to all parties when milestone battles affect the morale of the development team. Too often, the team sees milestone rejection as a personal rejection, each team member becoming angry at publisher for rejecting their individual work. Usually this interpretation of a milestone rejection is erroneous, and a project leader needs to isolate the team members from milestone disputes and concentrate the team's focus on the game's progress.

Once the developer agrees that the milestone needs to be re-submitted, he should attempt to get a very clear idea of what he needs to do to gain the publisher's acceptance of the milestone - not just what problems need to be addressed, but in some cases, how they should be fixed. However, if some required fixes adversely affect the schedule or budget, then the developer should so notify the publisher.

Conclusion

Regretfully, this paper casts the developer/publisher relationship as an adversarial one. In one sense, it is. The developer wants to be paid as soon as possible to cover his risks, and the publisher wishes to delay his payments to minimize his own. However, the best way to both prevent and resolve problems is by fostering communication and trust - not just between the project manager and the producer, but also between the developer's and producer's management. With open communication, followed by professional business practices and work that satisfies everyone's needs - not just what the contract requires - maybe that contract can stay locked up in the drawer a little more often.