NY-based developer Large Animal (Rocketbowl, Snapshot Adventures) switched to the Scrum method of agile development last year, 'sprinting' to complete individual game elements - here's just how it went.
[NY-based developer Large Animal (Rocketbowl, Snapshot Adventures) switched to the Scrum method of agile development last year, 'sprinting' to complete individual game elements - here's just how it went.]
Large Animal Games has been in business in New York City since January 2001. For the first several years, Large Animal developed games using informal, homegrown software development methods. "We did a lot of experimentation," says Wade Tinney, co-founder of Large Animal.
To track project schedules, for example, teams at Large Animal tried using MS Excel, MS Project, FogBugz, and even tried different visualizations of the project schedule using Adobe Illustrator and Visio.
Despite success with some of their practices, teams at Large Animal were still looking for improvements. Some of the things they tried had worked in theory, but felt forced. It was hard to motivate all team members to stick to some of these processes.
More critically, Large Animal was finding it difficult to grow, since a set of key team members were needed in certain roles on every project. While these key team members added a lot of value, they were a bottleneck limiting the number of projects that Large Animal could have running simultaneously.
In early 2006, Large Animal discovered agile development (and Scrum more specifically) and began incorporating some of the techniques into its project teams. Encouraged by the discussions about agile development at GDC 2006, Large Animal started holding company-wide meetings every morning.
As Wade describes it, "At first we had each person in the company talk about what they were working on. Over time we changed the format of the morning meeting so that the order that people spoke was grouped by project. As we added more people, we started having one person from each team report to the group."
Aside from this daily company stand-up, teams were still operating as they had been. The transition to a more complete implementation of Scrum started with a low risk pilot project that had a flexible time line. This project team would have their stand-up meeting, with a Scrum board, during the company-wide morning meeting.
This gave everyone an opportunity to observe, ask questions and comment on the process, which helped spread knowledge of Scrum throughout the company and helped share learning between teams when additional agile projects were started.
Today, all active projects at Large Animal use Scrum. The rest of this article describes some of the key successes achieved at Large Animal and some of the challenges that remain.
The article assumes that readers have a basic understanding of agile development and of Scrum in particular. For those readers who need a refresher on the fundamentals, Rory McGuire's Gamasutra article, Paper Burns: Game Design with Agile Methodologies, is a good resource.
The Impact of Scrum on the Organization
Even before adopting Scrum, project teams at Large Animal had always possessed some traits of agile development:
Software developed iteratively
Planning driven by bottom-up estimates
Comfort with the inevitability that "the plan" will change
Team-focused organization (around products)
This core culture at Large Animal helped smooth the transition to more formal agile techniques. Embracing agile development has impacted many aspects of Large Animal, enabling them to grow and develop as a company and to strengthen their relationships with publishers and other partners.
In its infancy, Large Animal was driven primarily by the talents and direction of a few key individuals. This arrangement worked while the company was still small and the number of projects was limited. As Large Animal grew, however, its ability to take on additional projects and still deliver at the same level of quality was constrained by the time availability of those key individuals.
Through the adoption of Scrum, the focus on quality that was previously dependent on a few individuals has been redistributed to the project teams. With some of the principle tenets of agile development coming into play, the team is incentivized to continuously produce working software that is focused on the highest priority user stories/features.
Large Animal has found that teams that are practicing agile need less guidance from senior designers and developers, thus allowing those people to direct a larger number of teams. As a result, Large Animal has been able to almost double the number of active project teams. More significantly, they have opened a new offsite development location in Atlanta, Georgia, a move that the founders would not have previously been comfortable with.
The level of interaction between senior designers (or senior artists or senior programmers) at Large Animal and project teams is partly driven by the skilled experience the designers possess, and it is partly a matter of trust. When the team is properly focused on the quality of user experience in a game and can deliver demonstrable builds at regular intervals, the senior designer does not need to be as involved on a day to day basis.
The agile development approach is good at defining a sandbox with certain boundaries within which a team can work. If the team strays off track, the iterative build-review process highlights where course corrections are needed long before the project is threatened.
Trust is also an important factor in the relationships Large Animal has with publishers and other partners. Take the experience on one of the Large Animal agile project teams as an example. On this project the publisher, as part of the product owner team, was involved in sprint reviews on a regular basis. The level of visibility that the publisher had into the progress of the team was extremely high.
This insight into what the project team was doing made negotiations with the publisher about the schedule impact of changes they had requested much easier.
From the publisher's perspective, the transparency of the development process and the progress of the team raised trust in Large Animal and lowered the perceived risk of the project. By reviewing a working build every sprint, the publisher had significantly more opportunities to impact change during the course of development than they had on previous projects.
"We take time to describe the development methodology to publishers even if they don't fully understand agile. I think it's comforting for them to know that we are not haphazardly building games, but that we are following best practices which were developed over many years and which are being used by many other organizations."
Wade also says that he's seen benefits of agile teams when it comes to the milestone contract structure. "The agile principle of continuously delivering working software means that milestone builds are never a surprise to the publisher. By the time we get to a milestone, the publisher has already seen working builds at regular and frequent intervals leading up to that point, and they've been invited to comment at each checkpoint. As a result, there is less pressure on the milestone and the milestone becomes more of a formality."
Project estimation at the start of a project is still challenging, but having a clear framework has made this part of the negotiation much easier. "We've been able to set the expectation that the milestone dates are not fully known or accurate, but with the reassurance that we have a clearly defined sandbox in which to work. The key thing is that we're not just saying, 'it'll be done when it's done'."
Wade also notes that release planning (projecting forward based on historical team velocity) has helped highlight issues early in a project and allows him and the other producers to adjust course before a situation becomes critical.
The Impact of Scrum on Project Teams
At this point, all of the project teams at Large Animal are using Scrum. Teams continue to experiment with different specific techniques and, as a result, end up doing things slightly differently from one another. However, there are some important common components.
From a high-level product planning perspective, Scrum masters (at Large Animal, producers fill the Scrum master role) use a product backlog board to lay out all of the user stories that haven't been completed and aren't in the current sprint. Each of these stories is written on an index card.
On the product backlog board, a member of the product owner team (usually a senior game designer, with input from a member of the publisher team) will arrange the user stories in priority order. The priority reflects the relative value of the story/feature within the final game. With stories arranged on the board this way, the team can approximate what work future sprints will likely contain and can project a release date for the game.
This exercise is useful for identifying red flags early, such as when the projected release date is two months past the milestone date agreed to in the contract with a publisher, or when a chain of dependencies is laid out that identifies a series of work that should be prioritized higher than it currently is.
(It should be noted that while the product backlog board is an important tool for reducing the long-term risk to a project, it is not used to commit a project team to a predetermined plan. The team only makes a commitment to complete those stories that they pull into a sprint.)
These high-level planning techniques are most effective after pre-production. During pre-production, the product backlog is still incomplete and it is extremely difficult to make schedule predictions that are accurate or useful.
Realizing the benefits of being able to react and plan against something built versus something that only exists as an idea, teams are Large Animal are trying to shorten the pre-production phase of projects and get into the production sprint cycle as soon as possible.
When the team moves a user story from the product backlog into a sprint, they physically move the card that describes the user story from the product backlog board to the sprint board. When this happens, the story is broken down into tasks that need to be completed in order to fully implement, test, and deploy the story into the game.
Teams at Large Animal find it useful to break stories into tasks as a team so that every team member can give their perspective on the work that needs to be completed to wholly realize the story. Each task is represented on its own index card and pinned to the sprint board.
The movement of the task card across the board marks the lifecycle of that task. As the task card moves it stays horizontally aligned with the story card that it belongs to. By arranging the cards on the board this way, team members can easily see their progress by looking at how the tasks of a story are placed across the board.
The sprint board is organized into columns: 'Story', 'To Do', 'In Process', 'Verify', and 'Done'. At the beginning of the sprint, the stories that will be completed during the sprint are laid out vertically in the 'Story' column.
Once tasks are generated for each story, they are pinned to the board in the 'To Do' column, next to the story that they describe. When a team member starts work on a task, they move the task card from the 'To Do' column into the 'In Process' column. Once work on the task is complete, the team member can move the task card into the 'Verify' column.
By moving the task into the 'Verify' column, the team member is indicating that they have completed the task and that they have tested their work. For most tasks for most teams at Large Animal, testing a task at this stage is an informal manual process. For a task to move from the 'Verify' column into the 'Done' column, it needs to be tested by a team member different than the one who originally completed the task.
Testing at this stage still tends to be manual, but is more likely to be based on a written test case or a quick discussion during the morning stand-up meeting (see below). When a task is 'Done' it usually means that the task is complete, tested, and the resulting implementation has been included in a working build of the game.
Teams at Large Animal recognize that there is room to improve the level of rigor with which quality is built into their games, and adopting a more disciplined approach to test driven development and integrating other XP techniques is one of their future goals. The sprint board shows how work during a sprint is tracked, but does not describe how the team works together to actually complete that work.
Teams at Large Animal typically use the following meetings to structure the collaboration and development that happens during the course of a sprint (the sprint preparation meeting and the mid-sprint meeting vary somewhat from team to team but are included here to give a complete picture of the types of interactions that take place during the sprint):
Sprint preparation meeting. Held a day before sprint planning. The preparation meeting is a time set aside for the team to get familiar with stories that have been added to the product backlog during the previous sprint and to assign story point estimates to those stories (via planning poker).
Once new stories have been estimated and before the planning meeting for the next sprint, the product owner will prioritize the new stories on the product backlog board. Not all teams use this meeting. Some teams combine the sprint preparation meeting with the sprint retrospective meeting.
Sprint planning meeting. Held on the first day of a sprint. The team identifies which stories from the product backlog it will commit to completing over the course of the new sprint during the sprint planning meeting. For each story that the team pulls off of the product backlog into the new sprint, the team breaks the story down into tasks that are each given more specific time estimates (in hours).
The time estimates on story tasks help the team ensure that they are not over committing. Teams can easily add up the estimated number of hours needed to complete all tasks during the sprint and compare that number to the number of hours available in the sprint.
Stand-up meeting. A 15 to 20 minute discussion held daily at 9:35AM during which the team updates their progress and ensures that all team members are in synch at least once a day. For most teams, updates to the sprint board are reserved for the stand-up meeting. Moving cards on the board only during the stand-up meeting ensures that everyone on the team is explicitly aware of each move.
Completion of the sprint backlog is the collective commitment of the entire team and the stand-up meeting allows each team member to have a clear picture of whether or not they are on track to fulfill that commitment. In addition to the sprint board, a burn down chart is also an effective tool teams use to visually illustrate their progress through the sprint (see Henrik Kniberg's Scrum and XP from the Trenches for more information on burn down charts).
Mid-sprint meeting. Held halfway through the sprint. The purpose of the mid-sprint meeting is to serve as an explicit checkpoint to ensure that the team is on track to completing their commitment. The team also uses the mid-sprint meeting to review the implementation of user stories with the product owner (the publisher or other customer representative). If any issues are identified, the team has an opportunity to make adjustments before the end of the sprint. Not all teams use this meeting, particularly not teams using short (one week) sprints.
Sprint review meeting. Held at the end of the sprint. The review meeting is an opportunity for the team to show the publisher/customer a working build of the game that includes implementation of the new user stories.
The teams at Large Animal place great importance on the sprint review as an opportunity to validate and challenge the direction in which the game is developing. The sprint review is structured as a formal presentation of the latest build of the game. In addition to the publisher/customer, everyone from the company is invited to attend the review and provide feedback.
Sprint retrospective meeting. Held at the end of the sprint. The retrospective (a.k.a. the postmortem) is a time for the team to reflect on what worked well during the recently completed sprint and what could be improved. The intention being that the team takes what it learns from every sprint and applies it towards improving the way they work together and improving the way that future sprints are executed.
Many of the process innovations developed at Large Animal originally came from ideas generated in retrospective meetings. Some teams combine the sprint retrospective meeting with the sprint preparation meeting.
Over this first year of using Scrum on projects, the teams at Large Animal have learned some valuable lessons and developed some unique enhancements to the Scrum process.
These tips and techniques have helped make the process more fun, humorous, and motivating:
Approach stories as a team. The basic idea here is to get team members to work on the same story at the same time during the sprint. If an artist, musician, and programmer all need to complete tasks to finish a story, try to have them all complete their parts to the same story at the same time rather than working on different parts of different stories.
Many benefits can be derived by working this way: individual efficiency and productivity increases since there is less time spent switching between activities and less time waiting on team members to complete dependant tasks in the story, issue resolution is more effective since the entire team will be focused on completing a story at the same time, and the risk of encountering an unpleasant surprise late in the sprint is reduced.
The 'no excuses' late box. As described earlier in this article, there are many points throughout a sprint where the team needs to come together. To help promote timely attendance to these discussions and limit the amount of time wasted by waiting for team members to show up, some teams have implemented a 'no excuses' late box. Team members who show up late to a team discussion are required to put a dollar in the late box. Proceeds from the tardy box can be used to help fund a morning breakfast get-together or another team building activity.
Punctual Scrum contest. Similar to the 'no excuses' late box, the punctual Scrum contest was implemented at a company-wide level at Large Animal to help teams start the day (with their daily stand-up meeting) on the right foot. Held over the course of a month, the contest gives points to project teams that start their daily stand-up meeting on time at 9:35AM and have all team members in attendance.
The team(s) with the most points at the end of the month are awarded a team dinner out at a restaurant. In an otherwise relatively relaxed and informal company environment, the leads found that full participation in the daily stand-up was important enough to team productivity that it was worth creating this incentive to encourage team members.
Using a calendar for the product backlog board. Some teams have found it useful to lay the product backlog board out on an actual calendar. With the calendar laid out, it is sometimes easier to think through schedule projections and what-if scenarios.
Question mark card in the planning poker deck. Some teams found that it was more straightforward to use a "?" card during planning poker on a story that was not adequately defined than to simply use a high value card.
Bag of life. Prior to creation of the "?" card, a team member who felt that they didn't know enough about a story to give it an estimate might have used a high value card. This sometimes had the effect of causing a panic attack in another team member. Hence the need for the bag of life (a simple brown paper lunch bag) to help with hyperventilation. While the introduction of the "?" card has reduced the need for the bag of life, teams still keep one around just in case.
Good conduct medals. As an additional incentive for positive team behaviors, some teams have created a set of humorous good conduct medals to recognize team members who demonstrate admirable performance.
Some examples include the Medal of Merit, awarded for completing all tasks during two consecutive sprints, the Bronze Verification Medal, awarded for getting all tasks during a sprint verified, and the Burndown Cross, awarded for keeping the burndown chart up to date each day during two consecutive sprints. More than simply recognition in word, these teams have designed actual medals that can be printed out and displayed by those worthy enough to earn them.
Open Questions
This studio's journey into the world of agile software development has not been completely intuitive. There are a number of questions and issues that they are still struggling with.
How to write a good user story. Philosophically, user stories are a perfect way to connect the day to day effort of the team back to its benefit for the player. Often, it's very easy to imagine what a player might want to see in the game (i.e. "I want to see cool particle effects whenever I do something good with my character.").
In other cases, however, there is work that does not fit neatly into the user story format. These are things that no user would ever ask for directly, and are only connected to a user benefit in a very roundabout way.
Toward the end of a project in particular, as the team is dealing with lots of highly focused pieces of work, it can sometimes reduce the clarity of the Scrum board to state all stories in terms of the over-arching benefit to the user. Teams at Large Animal have found that as a project enters the final third of its development that many user stories end up looking more like more like "super tasks" that encompass a smaller set of sub tasks.
Stories, tasks, or bugs. Prior to adopting agile, Large Animal was in the habit of using FogBugz as both a task and bug tracking tool. While this tool lacks the visibility of Scrum, it is very effective at helping to manage large numbers of detailed bug reports and reduces the chance that bugs fall through the cracks, especially on days where a build is undergoing heavy testing and revision before going out the door.
Unfortunately, there is often considerable overlap with work that is more loosely specified on the Scrum board and would be dealt with during the verification process. As a result, there is sometimes confusion about which tool to use to track an issue.
Time estimation. There's just no getting around the fact that it's really difficult to accurately estimate how much time it will take to come up with the right solution to a problem, whether it be in code, gameplay, or UI design. While agile methods have done much to make development more comfortable and predictable at Large Animal, estimating the time needed to complete a task is an ongoing challenge.
One team uses a four-week calendar board like this to sequence user stories within a sprint.
Closing
As you can see, the teams at Large Animal have taken the basic principles of Scrum and applied it to their projects in a way that suits their needs. Whenever necessary, they've filled in the gaps with homegrown techniques.
Even though they are still wrestling with certain issues, the agile framework has served to keep these problems visible and gives the team regular opportunities to discuss solutions.
This is the most important lesson that the Large Animal team has learned from agile; that they need to keep thinking creatively about how they work together and continuously try to improve their process. This mindset is the key to high performing, self-organizing teams.
Read more about:
FeaturesYou May Also Like
