Q&A: Avoiding crunch in live ops game dev with producer Grant Shonkwiler

Producers play a critical role in keeping game development teams happy, healthy, and productive. Now that so many games are designed to be played and updated long after launch, a good producer can do a lot to streamline the work of supporting a live game.

Here, veteran producer Grant Shonkwiler (who's spent time at Epic, id Software, and Megatouch) chats with Gamasutra about what it takes to be a good producer on a live game, and what you and your team can do to avoid crunching on your next milestone.

What's your background and experience with live ops in game dev? 

Shonkwiler: I worked on the first launch of Fortnite into alpha, until it was in beta when the servers were on all the time.

Since leaving Epic I've worked on a few smaller mobile games with clients that have had a live ops component. I'm not the most experienced person with this, but I'm learning everyday from my clients. 

Have you ever crunched (however you define it, though we typically start at "working more than 40 hours a week because you were asked, expected to, or afraid of the consequences") on a live game? If so, why?

I define crunch in two ways. Crunch = working more than 40 hours a week, for at most two weeks, voluntarily. Overwork = working more than 40 hours a week, for more than two weeks, voluntarily OR asked to.

I have done both in my career. I've not done it as much on live service games, again possibly due to my lack of extensive experience. The times I have done so have been to support the player base, usually it's not on new content as much as it is working on stability. 

You're not the first person I've heard use both terms, though definitions seem to shift between groups. Why is it important for you to define crunch vs. overwork? Is there a better way for studios, leaders, and devs to talk about this phenomenon?

I don't know where I heard this before. I know Keith Fuller uses the term overwork. I find it important to define the two for a few reasons.

First, people will want to work hard for short bursts, this can be incredibly useful both for the project and the team. I have found in super short bursts it builds trust on the team and moves things forward. But you need to make sure that you give some time off to recover, the goals and the timeline need to be clear.

For example, "Hey team we're thinking of pulling a one-week bug bash, some of us might work a bit late and we will make sure to have food. No pressure. At the end of the week everyone is expected to go back to working 40 hours max." It's a delicate thing and you need to find balance on your team. Also some people just like to work more than 40 hours a week, you just need to make sure they aren't burning out. Overwork is never useful. Being required to be in the office more than you need to be is burning out people and causing mass brain drain in our industry. There is a reason seven years in the industry is considered Senior. I do everything in my power to make sure overwork doesn't exists where I work. 

As far as talking, the main thing is that we NEED to be talking about it at all levels. Defining it, talking about what it does and doesn't do etc. Then when we need to move into crunch, make sure the team knows why and what is expected of them. Eventually we can remove overwork from our industry.

Have you ever worked for a significant amount of time on a live game without crunching? Did you or anyone else feel any pressure (from within or without) to work harder on the game?

Yes! Most of my time on live service games has been without crunching. There is always pressure to work harder, it usually comes from within yourself to do the best you can, it also comes from players wanting more content.

The job of a producer, like myself, is to help people understand what those pressures are and where they are coming from. It is then our job to work with team members to decide if working harder is actually the right solution at that time and for that person. Smarter not harder! 

What's the key to sustainable, effective live ops game dev without crunch, in your opinion?

The two major challenges of a live service game are supporting the live game and releasing new content. Each of these challenges require unique solutions. Let's talk about them each. 

Supporting the live game: This is the hardest one, players don't take breaks, when you have thousands or even millions of people playing your game they are playing 24/7/365, so you have to make sure your players can get the help they need as quickly as possible.

This may mean having different shifts, or people on call at times. This is a scheduling problem, making sure people are working the shifts they want, getting breaks/shifting shifts at times. And making sure people take turns on call, and after their time on call get some time off. An example would be "On call for 3 days a month, then you get a 4 day weekend after your time on call." It also means you need to have people whose main responsibility is knowing when a problem needs fixed NOW or when it can wait until later (the next day, or even the next release). These people also need to be on a shift schedule. It's important to also have a solid path of escalation.
 
New Content: This one breaks down into two parts. Helping players understand expectations, through Community Management and Marketing. And through making solid schedules. The second part is basic game development. Schedule sprints and releases, make sure teams aren't expected to release something every time.  

How do you go about designing an effective schedule for game development?

I break scheduling down into 3 levels: project, release, and sprint. 

Project is a high level schedule for your entire project, it doesn't have specifics, it just has milestones, releases and goals. So by Alpha 1 we are gray box locked, by Alpha 2 we are level locked, Beta we are Zero Bugs Recorded. Etc. For live service these will be Content launch 1, or 2 etc

Release (it's called release but it doesn't need to be associated with an actual release) is a feature level schedule for the next 3 months. Why 3 months? Because I have found anything more than that is a waste of time, it won't survive contact with players OR devs. So at the release level you have every feature planned out where it will slot in and who would work on it. You define here the teams, their work cadence and the actual release cadence. 

Sprint can be anywhere from a week to a month in which you define every task within each feature, get estimates and make sure there is time to complete things. Found work is moved off, or replaces work that already exists. Over time you and your team will improve on estimates and make sure you aren't over committing on sprints. 

Any specific production practices, tools, systems, etc. that you would recommend to other game makers working on live games?

The best practice I have heard of is making sure you schedule releases properly and allow for content that isn't ready to slip. Example: We have 3 releases coming up. We have 12 pieces of content we want released across those 3 releases. We have 2 teams working on these 12 pieces of content. So you set it up this way. Release 1, Content 1-3, Team 1. Release 2, Content 4-8, Team 2. Release 3, Content 9-12, Team 1.

This allows for teams to have time to work on things without the pressure of having to be part of every release. Also if something can't make it into the next release, it's not a huge problem for it to shift to the next release. 

You need to be very tight on what is in your sprints, what is being added in after the sprint has started, and making sure your estimates are being held closely. Make sure to build in time for iteration and bug fixing.