[Production consultant Keith Fuller surveys the industry to find out what producers really think about their studios -- both their strengths and what's holding them back -- and shares the results with Gamaustra, drawing compelling conclusions.]

I'm Keith Fuller, a production consultant for video game developers. After 14 years in the industry as a programmer, manager, and producer, I decided to start my own business to help as many teams as possible get better at making games -- with the goal being to hit your deadlines without resorting to overtime.

Obviously, it's in my best professional interest to learn more about where studios need to improve. But from a personal perspective, I've also wondered for years why there isn't more public knowledge sharing in the realm of production practices.

There are all sorts of venues available for programmers to better themselves and for artists to improve their craft in skill-specific workshops and online forums. Why has the project management aspect of game development been neglected by comparison?

As a first step toward addressing all of the above, I decided to offer a brief, anonymous survey to production personnel throughout the industry and find out from them what their company processes are like and what steps could be taken to improve them.

I advertised on my company website, my personal blog, Twitter, Gamasutra, Facebook and on the IGDA production mailing list, inviting anyone and everyone to take part, from production coordinator up to studio leadership.

No restrictions were made or implied regarding genre, publisher affiliation, geographic location, or any other distinguishing factor. As an incentive to garner more participation I even offered a free studio consultation to one lucky respondent. (See the Postlude for more on this.)

The results of most of the survey questions are presented herein for public review, as I promised in my advertising. I make no claims at any level of expertise in the realm of survey analysis, so feel free to ignore my conclusions and draw your own. But I think many of the findings are fairly clear and I hope that they generate meaningful discussion, preferably leading to industry-wide improvement in how games are made.

Process Improvement

To help people get into the right mindset I prefaced this section with the following:

For this question, consider "process improvement" to mean any single act or recurring practice designed to increase quality, reduce delivery time, reduce waste, or lower costs.

Basically I was trying to find out how many people are out there purposefully improving how they do what they do. I then asked:

Question #1: Do you actively engage in process improvement for a particular project, for the company as a whole, and/or routinely?

More than 83 percent of the participants said they actively engage in improving processes for a particular project. Of those, about half also improve processes for the whole company while about two-thirds also engage in improvement routinely. Further, of the 17 percent who don't seek improvement on a per-project basis, almost all of them try to improve things in some way at their company.

To restate those findings, more than four out of every five people in production are trying to make their current project work more efficiently in some way. Half of those same people are working on something to make the whole company operate better. And most of them are doing so on a regular basis.

The definition of "process improvement" was pretty broad here, but the results still indicate a very healthy number of people involved in production are routinely trying to improve their project and, to a lesser extent, their company. So these folks are out there not just performing standard project management -- checking that the designers are getting their art and the tools programmers are helping the animators -- but these producers are also looking for ways to improve how the people at their company do what they do.

Company Success

In this section I wanted to find out what producers think about their company's success. What does their company do well?

I started off with this explanation:

For the following questions, consider "company success" to mean the top-level goal of the current project or the organization in general -- essentially equated to profitability.

In your position you might not know how your work directly impacts sales figures, but you do know how success is ultimately defined for your project.

Example: If you're creating a DLC map pack for a popular FPS, your company success is likely to be based on timely delivery of high quality, robust content -- improve in those areas and you improve your company success.

Having established a definition for company success I asked:

Question #2: Name the area in which your company excels the most.

Perhaps unsurprising given the respondents I pursued, Production was the favorite among the eight options, taking home almost a third of the votes. Intriguingly, the first runner up was Innovation -- not what you might expect a quarter of the producers to say about their companies. Nearly one-sixth of the participants gave Work/Life balance its third place ranking -- another result I would not have guessed.

There were a few wiseacres who gave miscellaneous responses like "politics", but most everyone else divided their votes amongst Pre-production, Polishing, and Communication, leaving all three of those near the bottom.

In summing up the overview of these results, I have to point out the disparity between what the survey indicates studios are good at (Production, Work/Life Balance) and the bevy of war stories passed around about projects coming in late and even then only due to huge amounts of unpaid overtime. I'll leave it to the reader to ascertain whether or not the survey participants are the same people writing the postmortems.

What stood out most to me in this section was the fact that one quarter of the categories received no votes at all. Nobody who took this survey thought their organization excelled at Company Leadership or Mentoring. Keeping in mind the body of respondents, it's conceivable that fostering the personal improvement of employees might go overlooked (although any producer working with associate producers should take a hard look at this area).

So while I don't intend to minimize the importance of Mentoring in an organization, I'm prepared to let those results slide. But I want to stop and look at Leadership for a moment.

Even Work/Life Balance (a.k.a. Quality of Life, or QoL), the public perception of which indicates a subject of near-universal disgust in the industry, garnered infinitely more votes than Leadership. Why did this area get short shrift in these results? I'm going to propose a couple of theories.

Theory 1. Good Leadership is camouflaged behind any other area of success. In other words, you may not see what your leaders are doing behind the scenes; you only see the outcome as good production or above average innovation. But the quality that's evident in those areas is a result of the leadership that enables it. Personally, I'd really like to believe this is the case, mainly because I would prefer not to seem as though I'm badmouthing the leaders of studios anonymously represented in a simple survey. Nonetheless, I also have to propose the flipside of the coin which is...

Theory 2. Teams are achieving whatever their level of success may be despite sub-par leadership. Or the even darker possibility: teams aren't achieving success because of leadership that isn't up to snuff.

Game studio leadership is a tricky, tricky role to play. I'll be the first to admit that being the immediate layer of protection between developers and stakeholders is not easy. No part of my analysis should be taken to imply suit-bashing. Rather, I'd like to take a moment here to emphasize the importance of having quality people in leadership roles in every organization.

If you're a leader and you have peers, opening yourself up to their examination is key. If you're alone at the top of the pyramid, transparency and willingness to adapt and improve is even more important. Dr. John Maxwell said for good reason that everything rises and falls on leadership. Studio leaders, if you're not willing to continuously improve yourself, your people and your company will reflect that.

Quantizing the Impact of Success

From the number of people who skipped the following questions it seems I lost some folks at this point, but what I was hoping to do was move the participant from a fairly abstract observation ("What is your company's strength?") to an examination of measurable success ("How valuable is your company's strength?")

In my experience, if you really want to convince management of the importance of something you need to be able to express it in numbers, preferably prefixed with dollar signs. So I asked the following:

Question #3: Regarding your answer to the previous question, on a scale from 1 to 5 how much of an impact does that area of excellence have on company success?

Example A: My team rocks at Communication, but that really doesn't drive sales at all. I'll score it a 2.

Example B: My company is awesome at Innovation and that's why people buy our games. I'll score it a 5.

To rephrase what I was asking: go back to what you said your company was good at (recall that more people said Production than anything else) and now assign a value that shows how much that area impacts your company's bottom line. The results here are somewhat bell curve-ish -- as you'd expect since the available values were 1 through 5 -- but I'd like to draw your attention to a couple of points.

First, the more positive or beneficial upper end of the range had more responses than the lower half. Secondly, the specific average value here was 3.38. Whatever people think their company excels in, they feel it has a pretty good impact on overall success.

Where Would You Like to See Improvement?

This is the sort of thing that most participants were probably hoping to see -- that same list of areas from question #2, but now you get to pick which one you most want to see improved.

Question #4: In which of these areas would you most like to see improvement at your company?

There are two noteworthy points about the responses to this question:

So when given the chance to select any part of their company for improvement, somebody would choose any given area, but twice as many want to see their leadership improved. Remember in Question #1 when nobody said their organization excels most at leadership? It seems a fair number of folks would like to see that addressed.

I didn't provide a text field for people to enter a reason for their choice, so I don't know that we can glean many meaningful solutions from this. There is perhaps only one particular conclusion we can come away with -- leadership needs to be considered a key target for improvement.

Quantizing the Impact of Improvement

In an earlier question I wanted to put a measurable value on each company's area of excellence. Now I asked participants to do the same thing for their preferred area of improvement. I explained the value scale thusly:

For the following questions, please use a scale from -5 to +5. If you feel that a particular area is lacking for your team or your studio, it might actually be having a negative impact on company success.

It's really a two-part investigation. First, tell me the current impact of the area you want improved, then tell me what that area's impact would be if it was improved as much as possible. The first part was:

Question #5: Regarding your answer to the previous question, on a scale from -5 to +5 how much of an impact does that area have on company success (right now, without improvement) ?

Noteworthy here is that the average value was -1.64. Some respondents gave positive values, but most people said the area that needs improvement at their organization currently has a negative impact on company success (which is a point worth pondering in and of itself).

Now the second part of quantizing improvement:

Question #6: On a scale from -5 to +5 how much of an impact could that area have if it was improved as much as possible?

There were no negative responses here. If you were to improve anyone's selected problem area, the worst that would happen is you'd take a negative impact up to a zero. And the average of all the responses here was 3.36. If you combine the information gleaned from questions 5 and 6 you can see the following:

Obviously you don't want to read too much into the numbers on any survey, especially when you're averaging a set of responses, but just think about that last statement for a moment. If you take your average producer at her word and improve just one particular area of your organization as much as possible, you'll not only turn what's currently a negative impact into a positive one, you'll gain a 100 percent positive improvement on how that area impacts your company's success.

To make one last observation on this topic, let's revisit the response to Question #3 in which the participants were invited to assign a value to their studio's best aspect. How much did people say their company's success is impacted by what they do best? On average, 3.38. Compare that to what they answered in Question #6 when asked about the impact it would have on success if maximum improvement occurred in an area, which was 3.36. The values are nearly identical. To rephrase these results:

If you fix what needs fixing it will have as much of a positive impact on company success as the area in which you already excel the most.

For those who may have been on the fence regarding the value of improving their production practices, I would think this provides a compelling reason to seek out ways to better your studio.

Summary

This survey comprised ten questions and I've only delved into the first six in this analysis. I won't get into the remaining four just yet as they tended to be more open-ended, blue-sky queries. They were enlightening nonetheless, but not as easy to extrapolate general action items from as the numerical topics I've just covered.

I hope to leave the reader with a big picture view of the importance of improving how your studio does what it does. This survey invited production personnel from studios of all sizes to give their responses anonymously on a wide array of topics impacting the efficiency and success of their organization. To generalize the average feedback, the participants indicated the following points.

Several meaningful questions arise from these points. If so many people are already striving to better their studios, why the bevy of negative responses about current company status? If the need for improvement is clear and the benefit of improvement has great potential, what's preventing a solution from being implemented? What steps can be reasonably taken to address deficiencies in areas such as Mentoring or Leadership?

Some of these questions are answered in the remaining 40 percent of the survey that wasn't covered in this analysis. I plan on reviewing those responses publicly at some point in the future. For now, I'm happy to leave these questions and others with the reader and I invite their discussion. As a production consultant it's my goal to help companies improve in all of these areas, so feel free to send me email or follow me on Twitter as I seek to uncover and implement solutions for the benefit of the industry as a whole.

Postlude

I'd like to thank all my friends and colleagues at various studios and throughout the IGDA for their assistance in creating this survey. I wanted to make sure it would be accessible to as many people as possible and that it would generate meaningful findings -- you all were particularly helpful in that regard.

Also, thanks are due to the production folks who took time out of their day to answer my 10 questions. I realize that taking a survey is rarely a favorite event for anyone, but whatever benefit is eventually derived from this venture wouldn't have been possible without your contribution.

Lastly, I'd like to specifically thank Hunter Hudspeth, associate producer at ZeeGee Games and winner of a free studio visit from Fuller Game Production. I look forward to meeting you in person and hope we can work together to find ways to improve your studio's production practices.