Manager In A Strange Land: Milestones Considered Harmful?

We didn't have milestones for Spider-Man 1. Well, we did…we had a schedule of month-by-month deliverables with lists of items that were supposed to be delivered by when. But we rarely looked at that document during the project. And nobody from upper management ever said, "Why isn't item A, which was supposed to be done this month, not done?" And yet we still managed to ship on time.

In Critical Chain, Eliyahu M. Goldratt suggests that one of the things that makes projects go south is the padding put in between milestones. Because when you pad a deliverable, and you finish that deliverable early, you don't move on to the next deliverable. Instead, you polish that deliverable beyond what it needed. Gold-plating. Tom DeMarco says the same thing in Slack.

DeMarco and Goldratt say that instead of having deadlines, we should have optimistic goals. We will almost always miss these optimistic goals. But we won't lose any time to gold-plating.

But what if something goes wrong? (Which happens over and over and over on a game project.) If you make optimistic goals, you'll probably miss them. That's why, instead of putting the padding between the milestones, you put the padding at the end.

Most game development contracts have this padding built in, in the phase most of us call "alpha". We try to finish the game early, so it can get into testing, so we can fix the bugs.

I hadn't actually read these books when we were working on Spider-Man, but I think they help explain the phenomenon we saw, which is that you don't actually need milestones per se to ship a product.

So how big should that pad at the end be?

I've frequently seen milestone schedules that factor in one month for "alpha" and one month for "beta". (The difference between "alpha" and "beta" was never really clear.) I don't believe this is enough. This might be enough time to fix the bugs in a twelve-month-long project, but not for anything longer. This padding isn't just to fix a few lingering bugs; this padding also gives you some slack so you can miss your target and still ship on time.

Also, the number of undiscovered bugs that are introduced during the lifecycle of a project is proportional to the length of the project. With Spider-Man, an 18-month project, we tried to be content-complete four months before the ship date. (Although we slipped; the last in-game cutscene limped in with only a month left.) Still, the bulk of gameplay content was finished within a month of that target date, so an early alpha served its purpose: giving us room to slip. We even had enough of a comfort zone in those last few months that we could add a few small features.

What if we had a cast-in-stone milestone schedule? When that first set of levels was supposed to come online, we would have missed our planned date by a great deal. In addition, we probably would have decided to cut the amount of game content in half -- or more -- to get back on schedule. So we would have created a smaller game. (And it was pretty small to begin with - some reviewers said it was too short.)

Also, we would have lost flexibility in our development: the opportunity to find better, faster ways to do things can be lost if you don't have some slack time.

There are downsides to the "goals, not promises" method of milestones. First, it hurts morale when you keep missing your goals, so you need other methods to keep morale up. These morale-boosting exercises include telling everybody how awesome they are all the time, and "love ins" where you show the game to the team and show the progress that's been made lately. The other downside is you almost never punt. When you're approaching a cast-in-stone milestone, and you're about to slip, you can alter the deliverable to achieve the milestone goal. For example, your publisher says you have to deliver two levels, but you only have one, so you cut that level into two parts and hand it over. That's an example of bad punting, but sometimes punting can be good. You reduce the scope and deliver something to spec, even though it's not what you originally wanted to deliver.

Finally, "goals, not promises" milestones encourage a phenomenon I call the "Dead Zone", which I'll get into next article.

Still, let's say you're on board: you want to do "goals not promises" in your milestones, but your publisher does not. So if you don't hit your milestones to the letter, you're not getting paid. What do you do?

Greg John, producer on Spider-Man 1, has this message taped to the wall of his office: Minus Promitte Nivus Perfice. That means "Under promise, over produce." If your publisher is going to punish you for missing milestones, you have to pad the crap out of them. But then you'll probably fall into the gold-plating trap - you'll work on that first milestone deliverable until it's a highly polished gem, and when something goes wrong on the next milestone, you won't have any slack.

To make that padded milestone schedule work, you have to have internal goals that keep you well ahead of the external milestone schedule. Greg John and I did this when we worked on Tony Hawk ports. Early in the project, we were weeks ahead of our milestones. As the end of the project drew near, that gap narrowed. These were six-month projects, so for a longer project, you have to have a huge gap-I'd say try to be two to three months ahead-in front of the first milestone.

It would look something like this:

If you don't hit the internal schedule, it's not the end of the world. But it does mean you have to cut features and struggle to catch up, because you have to try to get that padding back in case something really goes wrong. Because it will.

If you haven't felt what it's like to have the burned disc in your hand for a milestone that's not due for weeks, I highly recommend it.

A side note: if you do actually manage to hit your content-complete goal, this gives you an option that game developers usually don't get. You can add wish-list items and polish. Gold plate all you want…at the end. Enjoy.

One last thing: by the time you're reading this, you've probably already given an optimistic milestone schedule to your publisher. You may be tempted to put padding in by shrinking the internal schedule. You can't. Your best recourse would be to cut features and renegotiate the milestone schedule.

______________________________________________________