The Battle of the Gut

Do you design games based on feel?

Do you take pride in your knowledge of core mechanics of the genre?

Do you measure your game knowledge based on the number of games you played?

Does the prospect of copying another game’s design turn your stomach?

Does someone asking you to justify your design opinion raise your hackles?

Do you favor originality over commercial success?

If the answer to some or all of these questions is YES, then maybe, like I once was, you’re a gut designer. As gut designers we may have made some successful games, but we’ve also shipped or eventually will ship a few duds. It is inevitable because we’re not always right. Design opinions and instincts are largely subjective.

If you are not a game designer yourself, you may have witnessed a design debate with a gaggle of game designers. We can be a very pretentious sort! Game designers are creatures of opinion. That is the reason we are hired. We are the gamers and subject matter experts that are entrusted with the heart and soul of the game. Yet this is ultimately a business with commercial goals, not just an art form for self-expression. This design-by-gut has to stop or at least be tempered with data analysis and business intelligence.

Background

In my 30 years making and publishing games, I’ve seen a lot of changes - from 8 Bit 2D games to 64 Bit 3D and VR games, from single player to massive multiplayer games, and from game cartridges and discs to downloaded games. It’s this last change that also opened up the possibility of new business models and revenue streams. We were no longer limited to hard-goods retail sales but could make money with so called free-to-play games with microtransactions and ad revenue. These ongoing revenue streams have made their way back into the console and PC games and moved publishers to embrace the idea of games as a service with a constant need to update the game to retain players and improve performance. This has made business intelligence and analytics extremely important. This also made the role of Product Managers critical to a game’s success. 

Intro the Product Manager

I am no longer a gut designer. I am currently employed as a Senior Product Manager. Back in the not too distant past, product managers existed only in marketing departments as they dealt with advertising, packaging and the sales channel. I’d be the first to tell you that I am not a marketing person and don’t plan to be. So what does this role have to do with game development?

This is a product development role that popped up in just the last few years at every major game publisher, and yet it hasn’t made its way into every game development team or publisher. It’s a role borrowed from SAAS websites and other software companies. This evolution of team structure and roles isn’t a new phenomena. It was Electronic Arts after all that borrowed from the film industry when it introduced the “producer” and “game director” roles to game teams.

Like SAAS websites and commercial apps, many games are now released in stages according to a product roadmap. Each release is designed to retain and attract new players. This includes content packs but also some core improvements and new features. Free-to-play games or any game with microtransactions are also trying to convert non-paying customers into paying customers - what’s called conversion. These microtransactions for content, game currency or lockbox keys all have their own in-game and e-mail advertisements, which are part of an overall customer relationship management or CRM program. The product manager is chiefly responsible for this CRM program, the ads and the overall product roadmap.

Product Manager vs Game Designer

So how does this product manager role dove-tail with the game designer role and where is the overlap?

#1) Many designers also do level layout, stat entry and scripting or other forms of implementation that puts their hands in the source control system. Product managers don’t do that at all, or shouldn’t.

#2) But on a senior or director level, they both come up with design documents. Google, Amazon or Facebook will call these Product Requirements Documents (PRDs) while the video game industry calls them Game Design Documents (GDDs) or Functional Specifications. These documents universally define the WHY and the WHAT in the why-what-how-when questions that various documents and job responsibilities try to address. The difference is how they come up with their design and to certain extent the level of detail.

Competitive Analysis for Parity

Product managers use competitive analysis to identify what is working for others. Many of the design requirements are aimed at bringing the game up to parity. This is the area in which many gut-designers shirk the business goals of success and focus on the conceit of making original games, nothing derivative. I would argue with a metaphor - there are many ways of making a car but they all share certain functionality, and one doesn’t reinvent the steering wheel if they don’t want to frustrate or confuse potential drivers unless that design is really what sells the car. So a product manager first and foremost identifies the common features the game must have to meet common player expectations. Now how those are implemented may be where your game differentiates itself, but it should be with an eye to be more successful. Your design should never be different just for difference’s sake. Choose your battles. Invest in the real differentiators.

Metrics and KPIs

Product managers produce or design dashboards that report trends, current and target performance using business intelligence (BI).  BI involves defining metrics from data sources such as app store purchases, installs, logins, player progress and other behavior, UI navigation in the player funnel, population size, ad clicks, session time, and ultimately some very important measures for games like revenue, conversion, retention, cost per acquisition and lifetime value. It’s these key performance indicators (KPIs) that drive design decisions for a product manager.

AB Tests

For those of us used to trusting our gut, it’s hard to argue about the common goals of increasing our KPIs, because indeed that’s the point of making improvements to the design. Instead of arguing with your gaggle of designers or pushing back on your bosses for their zany ideas, you can settle these design debates with an experiment. What you might not be aware of is the number of experiments you are subjected to every month from Twitter or Facebook as they look to improve their apps. This is because tests are conducted against a small subset of their users, called cohorts. Cohort A is the control or no new feature. Cohort B (often as little 5 to 10% of users) is seeing the new feature. Then after a few weeks of data, they compare the KPIs from A to B and make a decision based on the KPIs to either revert or commit the feature to everyone. Sometimes there may be multiple ways to implement a new feature, and you will have A, B, C and D cohorts. It can get pretty crazy, but with good statistics and confidence modeling, you can choose winners with a fairly limited subset of your players. Of course, this can get very expensive to do with every design decision. You wouldn’t want to do it with your parity features for your first release just the changes you are making to your game. Game designers, especially those of us who shipped games on cartridges and floppy discs where it all had to be perfect for the gold master cause you ain’t got any more chance to get it right, should embrace the fact that you CAN fix it and vett your ideas with AB Tests.

Design in the Details

So how do these two roles coexist on a team? Well for starters a product manager may only exist in the publishing side, and like all publishers should paint with only broad strokes as to WHAT they want in terms of goals. Very often, these broad strokes need to be broken down even further where it becomes more of a HOW question where the expertise of the game designers refine and interpret the ideas. So even if they are all in the same building, the product manager can refrain from being too specific and give some slack in the rope to the game designers and UX experts. This is how game designers can bring their unique talents and insights into the design feature to make it original and compelling, as long as it doesn’t contradict or step outside the box defined in the product requirements.

Advocates for Analytical Design

Yet another way some companies may divide the roles is to have a product manager focus more on CRM, metrics, AB tests analysis, ads and game economies while leaving the fun stuff like game design to the creative directors. Yet therein lays a flawed compromise, as creative direction done without analysis or ignoring the game performance considerations is doomed to fail. To avoid this, the product manager in these scenarios needs to be the advocate of analytics and what is commonly called business intelligence. Just like a game producer can push back on design from a budget or scheduling perspective, a product manager should push back on a design decision from a data analysis perspective, because if the new feature is doomed to reduce revenue or retention or be too big a cost for very little gain, then it shouldn’t be done. This is what us old school gut designers would call “bang for the buck”, and that’s how you should take that feedback from a product manager, rather than “shoot from the hip” and hope your gut serves you well.