The agile methodology known as Scrum is rapidly gaining development credence, and High Moon Studios CTO Clinton Keith (Darkwatch, The Bourne Conspiracy) presents this in-depth Gamasutra article explaining how publishers and developers can benefit through regular, focused iteration.
[The agile methodology known as Scrum is rapidly gaining development credence, and High Moon Studios CTO Clinton Keith (Darkwatch, The Bourne Conspiracy) presents this in-depth Gamasutra article explaining how publishers and developers can benefit through regular, focused iteration.]
Scrum, an agile methodology, is emerging as a powerfully beneficial toolset for building games. The approach of finding the fun through iteration rather than trying to plan and predict it completely up front is appealing to many developers who have repeatedly been surprised by all the unanticipated problems and discoveries made on the path to creating a game.
However, most publishers are too risk-averse to allow a team to not provide them with some form of detailed plan for the entire project. The developer using Scrum may resist this because they want to be able to adjust their priorities to the emerging game. This article addresses these issues of how publishers and developers using Scrum can work together with a long term plan and leverage the benefits of using Scrum.
One of the common misunderstandings of agile is that it avoids any long-term planning in favor of only planning a few weeks out. What agile does not do is support highly detailed long term plans up front. Agile methodologies, like Scrum, focus on continual planning for the entire range of the project. Like the iterations on the project itself, agile continually returns to the assumptions of the plan and modifies it based on reality.
One of the principles of Scrum is that the team commits to the amount of work they feel they can finish in small iterations of time called Sprints. These iterations are typically two to four weeks in duration. The results of the last Sprint will be used to potentially modify the goals and priorities of what will be worked during the next Sprint. This allows the goals of the project to "drift" a bit every few weeks. Ideally this drift is for the benefit of the game.
Among the adopters of agile in the game industry, questions are frequently raised about how this "drift" can exist when the publisher demands detailed plans. On the other side, publishers fear that without schedules, an agile team can iterate endlessly and never finish.
Fixed schedules attempt to predict the priorities and a rate of work that can be done ahead of time. Agile is a response to the reality that such predictions aren't very accurate and that we need to continually revisit our plans and adjust them during the entire project. Simply planning things out up front is not enough. The drift inherent with Scrum can be incompatible with these fixed schedules.
Customers often derive a sense of security from schedules. They want to be assured that a commitment is being made to deliver a product they can understand, with a known schedule and budget. It's not something they are going to abandon easily even though they are aware of the problems with fixed schedules. They have seen projects miss those schedules and budgets many times. They need something proven to work better if you want to introduce agile practices to long-term planning.
A challenge for the game development team using Scrum is to adjust the practices to meet the needs of their customers more closely while preserving the benefits of agile.
Many game developers applying Scrum have adopted what they refer to as a blend of Waterfall and Scrum. This usually means adopting Scrum practices like Sprints, Daily Scrums and Burndown Charts while maintaining detailed plan documents such as schedules and design documents for long term planning purposes.
The Sprint practices for Scrum are very easy to understand and implement while the Scrum practices for long term planning are not as easy to grasp. Some Scrum adopters or customers will not trust abandoning long-term schedules completely.
How well can Scrum teams work with customers who require schedules? It depends on the customer. If the customer demands a schedule high level of detail over the course of the project and will not tolerate variation from that demand, then the Scrum team will be less effective.
Some of the Scrum practices, such as the daily stand-up meetings to address impediments, and the regular delivery of a working game, will still help. However the team and product will not see some of the major benefits of Scrum. These benefits include the improved productivity of teams who take a level of ownership in their work and the improved dialog between the team and customers on the emerging game.
One major benefit gained from transitioning from deterministic schedules to iterative planning cycles is that the customers assume more control over the schedule and budget by manipulating the scope. This is done by continually prioritizing the list of features that they want in the game (called the product backlog) based on what value is emerging from the real game. The team delivers these features on a regular basis.
If the customer decides that the features built up are sufficient for the money spent or if the fixed delivery date demands, then the team can move over to building the production assets and completing the game. The benefit of this approach is that features of less importance to the game are abandoned before a great deal of effort is spent on them. This is in contrast to having the schedule or budget force the decision to cut key features that might be 90% complete.
Schedules often don't predict many of the problems which slow down development and don't prioritize the work based on value. Instead they try to optimize the resources to work in parallel and define when everything will complete at the same time. In reality, these parallel efforts rarely stay in pace with one another and the cross-dependencies can slow the entire effort down. This leads to team death marches and critical features being cut in order to meet a delivery date.
Clinton Keith is the CTO for High Moon Studios, located in Carlsbad, CA.
People refer to agile as a "management fad", but nothing is new about the practices agile wraps. Iterative and incremental development was the dominant way games were developed early on.
Dave Theurer, the legendary designer of Missile Command and Tempest, describes the highly iterative process for making arcade games in the '80s and how that changed:
"Pick an idea. Write up a game proposal. Get it OK'd by management. Take a couple weeks to bring up a playable simple version. Management reviews that and OKs it or axes it. If OK'd, continue with the whole game. Regular reviews by management to make sure still fun. Kill [the game] if not. Then field test and focus groups. Read collection reports. If makes big bucks, you're in there with a winner, and finish it up. Otherwise kill it or [make] big changes and keep going. That was a big problem later on at Atari: they forgot how to kill projects that weren't fun, and would let them go on forever."
I experienced the iterative planning cycle in the mid 90's working at Angel Studios, which was a member of the Nintendo Ultra 64 Dream Team. Nintendo would discuss a game idea with us and ask us to "find the fun". They'd give us enough money to operate for three months and then come back at the end of that time to see what we found (occasionally Miyamoto would visit). Usually we discussed the results and the high level direction for the next three months. Occasionally, if we couldn't "find the fun", the game would be canceled and an entirely new game would be started.
Most publishers acknowledge the problems with trying to create detailed schedules and plans. As a result, many development contracts these days acknowledge this by having a rolling level of detail for future milestones.
An example of this would be a contract that specifies a high level of detail for the next milestone with the level of detail dropping for milestones that are further out. As the project progresses, upcoming milestone details are fleshed out with more detailed promises of what the team hopes to deliver.
These "Rolling Milestone Schedules" start to look a lot more like the Sprint cycles in a traditional Scrum project. They still offer little room for the backlog to change within a milestone/release cycle, but the team can grow the amount of change the customer is comfortable with by growing their level of trust and confidence with the practices.
In order to allow more change to occur during the milestone, a development team has to address the relationship with their customers, especially the publisher. Let's start by looking at how publishers often view their external developers. There are a number of tacit assumptions that can exist on the publisher side:
We have to have detailed schedules so that the developers will work hard and we'll get our money's worth out of them.
The only control we have over the developers after the contract is signed is the milestone payment, so we have to get all our details for the game in up front.
Because the developers can't afford to miss milestone payments they are not open to our change requests during development, so we need to get all of our requests into the design documents.
This does not foster a very collaborative working relationship between the publisher and the developer and it's not going to change overnight. The customer and developer need to build trust slowly. Fortunately, the principles and practices of Scrum are designed to build trust. For example, at the end of every Sprint there is a Sprint Review where the team and the customers can play the latest version of the game and influence the priority of the work that can be done during the next Sprint. If the customers attend these reviews they will see a number of benefits:
Improved velocity of progress every Sprint.
Regular working versions of the game they can play.
The ability to offer suggestions and feedback that is taken seriously by the team and affects the next Sprint demo they will see.
Artifacts in the form of the task-boards and burndown charts that demonstrate daily progress.
While you may take away some of the sense of security of the customer without detailed schedules, you give working versions of the game on a frequent basis that demonstrate -- not predict -- progress. This should help them retain their sense of security.
The largest benefit to the customers is an increased level of productivity and flexibility. As the team improves its velocity, it should have some slack built in to add things that the publisher might want to see added or improved in the period of a few weeks. This benefit can become the biggest selling feature for Scrum and encourage the customers to allow more slack in the immediate milestone definition for even more change. Once the cycle of feedback and adoption/demonstration of changes is established, then trust grows.
One of the main differences with game teams using Scrum is that we don't release a "potentially shippable version of the game" every Sprint. The reason for this is that games must ship with around 10 hours of gameplay. This requires a great deal of production assets to be created and tuned.
Since production assets are highly dependent on game mechanics and technology, most of these final assets are not created until we know exactly what the game needs. As a result, game teams using Scrum still divide their projects into two phases: pre-production and production. This avoids wasting a lot or time and effort producing the wrong assets too early.
The production of a minimum number of hours of content is a major constraint on how iterative and flexible long term planning can be on a game development project using Scrum. Game projects are often faced with a 12+ month production "debt" of work to do and a fixed shipping date based on holidays or movie release dates. The combination of these two often leads to the demand for long, detailed schedules.
One of the problems with long term schedules is that they try to plan away uncertainty. Uncertainty can take many forms, but generally falls into three categories:
Uncertainty in what you are building.
The exact specifications of what makes a fun, hit game are impossible to document and schedule. "Finding the fun" includes many iterations and experimentation.
Uncertainty in technology.
Game developers face a rapidly changing technology base with products that require major innovation. Many games have been delayed or cancelled because the predictions of what the new technology could achieve or how long it would take to be implemented were usually optimistic or uncertain.
Uncertainty with the developers.
Given the same design document and schedule, two different teams will have entirely different results and levels of success. People aren't cogs in a machine. Teams gel differently and talent is highly variable.
Agile methodologies were created for use in cycles of product development that have a high level of uncertainty. Uncertainty increases the further out you try to predict the future, or the larger the task you are trying to estimate (see Figure 1).
The benefit of agile is in iterating on short-term, detailed estimates of short tasks. Agile limits the detail of planning to the matching of certainty and priority. When we have a lot of uncertainty in high priority features, we break them down into smaller parts that fit into an iteration. This way we can continually refine the estimates for the whole to be increasingly accurate.
Figure 1
As mentioned earlier, game projects are often divided into pre-production and production. If you want certainty in production schedules you need to address the three areas of uncertainty during pre-production. This has to be the goal of pre-production and the measure of when pre-production is complete. Ideally pre-production will demonstrate a certain percentage of all asset types combined in a shippable demo version of the game that demonstrates the final fun factor, performance and resources required to build the remainder of the game.
Some Scrum adopters in the game industry have adopted an iterative approach to pre-production followed by a detailed production schedule. This is effective, but it does not mean that all the benefits and practices of agile should be dropped when the team enters production. The principles of agile can still help in production.
There are benefits of agile that can be employed to help the team and the customers refine the production schedule:
Iterative production estimates even in pre-production.
Sprints deliver working software with potentially shippable assets. The team can use the empirical information of the true cost of producing these assets to refine their estimates of making all of the remaining assets of the same type. For example, if the schedule anticipates nine months of production, the team should start verifying that schedule 18+ months in advance of shipping. You don't want to discover that you really need 12 months of production when all you have left is 9.
A production schedule is just a prediction. It needs refinement.
Once a schedule for production is in place it should be treated like a prediction given the facts at the time it was created. This prediction should be continually examined and refined. The team should find ways to reduce the cost of production while improving the quality of the assets. More time freed up from production can be dedicated to polishing and tuning during post-production.
When you give visibility and control of the feature set to the customers, there are some problems that you should watch out for.
The customer who doesn't always know what they want.
As Spider-Man's uncle said: "with great power comes great responsibility". You are giving unprecedented power to the customer with Scrum. Not all customers know what to do with this new power. Make sure that the customer understands that every decision has an effect on the entire project. A pet feature that is being prioritized over other features will result in a more worthy feature being dropped later in the project. Scrum allows you to focus test early. Do that with the customer and see if your true customer really enjoys the game or not.
Customers who don't always agree with one another.
Sometimes the customer that shows up at the Sprint planning sessions isn't the one you really need there. They may not share the vision of the rest of the customers and they might give the team feedback that can send the product off into areas that the other customers do not want it to go.
Scrum's solution to these problems is embedded in the "Product Owner" (PO) role. This is the person who has the final say over all customers' input. This role is difficult to assign to one person on either the publisher or studio side.
A Product Owner from the publisher may not know enough details of development to properly prioritize some of the work that the team needs to do. A studio PO can be more responsive for all the customers that want to know about the vision for the game. This role is closer to a Project Director role than a traditional Scrum PO role. A studio PO can lack sufficient authority to override all of the customers, however.
If you are a customer of the game (especially on the publishing side), there are things you need to do differently from traditionally run projects to avoid problems specific to agile:
Avoid the "too many parts on the floor" warning signs.
You and the team may agree on a vision for the game, but don't count too much on the future of that vision. Avoid excessive parallel development of multiple features where possible. If you find yourself saying "all these new things are great to see, and they'll be fun when they all come together in the future", then you may be planning too far ahead. You should expect some loose ends every Sprint, but you should also see the game getting more fun as well. Not every Sprint will be an improvement, but most should be. If the game continues to be iterated with problems that remain unsolved or parts that are not being assembled to make "more fun", then you need to demand that the team address these. This is your responsibility as a customer for an agile team. You can even request that they spend a Sprint just assembling the parts together.
Don't replace a document with a plan in your head.
Scrum is not a tool for you to micromanage a team. It clearly divides team ownership from the product ownership. As a customer you have to balance your vision of the game with the results that are being produced every Sprint. If your vision for the game is not panning out with the game that's actually emerging, it's time to ask yourself some tough questions about your vision.
Being too distant or uncommitted.
Customers need to participate in regular reviews of progress and iteration planning. It is not merely beneficial, but necessary, to the team's success. If you are too distant or have not committed to the regular cadence of the review and planning cycle of agile, then don't assume you are going to be pleased with the game when you finally see it months later. The team and customers rarely share the same vision of the game from the start. By understanding and influencing the small steps of the project, you remain committed to where the project is heading. If you cannot make regular trips to see the game, have the studio PO visit you. Receiving just the build is usually not enough. If the team is too far to send someone regularly, try to get them on a conference call with the latest build running at both locations and discuss the game.
Communicating a vision for the game.
Communicating the product vision within the team is hard enough. Communicating it between customer and team is even harder. As the customer for the product, you need to communicate about the goals for the project. This involves a clear understanding with the marketing team and every other stakeholder on the customer side and the team as well. If you don't do this you allow the team and the studio PO to define priorities different from your own. This may result in the development of a game you did not plan for. You also risk having the team retrace missteps, and waste time and money.
If the barriers for all the customers to regularly review and plan is too great, then there needs to be a single customer whose responsibility it is to represent them. As with the "studio PO", this "customer PO" can be the conduit for all the others. This role is closer to the traditional Scrum PO. With these two roles, the customer PO and the studio PO communicate on a frequent basis about the game and the backlog. The customer PO may have the last say on rare disagreements of priority between the two, but they should share a common vision.
Planning and Scrum are not incompatible, but the particulars of when to plan and how much to plan at once change quite a bit when switching to Scrum. Publishers and developers cannot replace detailed plans and schedules with anything but frequent communication.
Long term agile planning has been greatly enhanced by the practices that Mike Cohn has introduced in his book Agile Estimating and Planning, and in the courses he provides through Mountain Goat Software. These practices give teams additional tools to apply in predicting how much work can be done further out in the future, while still allowing an agile project to introduce change.
The bottom line is that Scrum is a set of principles and practices that can support great teams making great games. The practices are meant to be adjusted to the team, customer and product where it is being used. The team and customers should get used to iterating on how they work together in the same way they iterate on the game. It can take years to achieve the full benefits of Scrum, but you should start seeing some benefits right away.
Read more about:
FeaturesYou May Also Like
