Fight Design Bloat with Data

There's a saying in advertising that 50% of the money spent on any campaign is wasted. The only problem is that you can never know which 50%.

The games industry lives in much the same state. There are a huge number of constituent parts included in a modern big-box videogame and most of the time drag involved in those projects comes from spinning off in a dozen directions and then reconciling them all into something. And, often, then realising that what the project has reconciled into has serious problems so it must once again be unpicked and rewoven.

The other side of the advertising story mentioned above is that in the modern age, metrics help to find where that wasteful 50% exists. The reason why Google is a multi-billion dollar corporation (and Facebook likewise) is that it provides tools to advertisers by which they can measure what works and what does not, with the net result being that much of what was the advertising business and industry culture is dying. It is *proven* to not work as well as the people inside the walls of the industry argue that it might. 

The thing is: 80% of those parts included in a modern game are unnecessary. In any successful game, the reason for why it is successful usually boil down to a few key things - and what defines a great game versus a rubbish game is how those few key things behave. All the other parts are generally just sitting there, not paying for themselves, and with most of the team arguing the "what about the player who does like X" minority interests.

As a current example, Bioware has shown through stats that 80% of Mass Effect players just play the default male character, and most of them play the Soldier class, and only 50% of all players ever bother to finish the main quests in the game. Doubtless many will argue that the game should serve the 20%, the minority and the 50% anyway in order to capture the greatest overall number of players, right?

Wrong. 

"Keep It Simple Stupid" is not just a bumper sticker slogan. It means working very hard to find the most efficient design that users will actually use, not including a variety of pet projects as part of a bloated software project on the idea that someone somewhere might find that one feature useful once. That's just ego talking. The hard job of game design is knowing what to cut and what is an *efficient* use of team resources to make a great game. Once you manage to do that, and do it well, the audience for your game will geometrically increase while the cost to produce it will dramatically fall. 

Measurement is the best way to cut to the heart of what actually works and what does not. It cuts through all of the ego in one fell swoop and shows, baldly, what is working and what is not. Done early enough and often enough, measurement tells you what you're doing wrong. 

Pure test-driven-development and design is only good for telling you how what you have is working or not. You do still have to do the hard work of making the next leap, the step beyond and the next stage. You still have to be an artist because tests can't act as a replacement for creativity of course. 

They also can't act as a replacement for courage. It is a part of the tradition of development that it is a team effort, but that sentiment to find agreement among parties is often the source of a raft of included features that are included because nobody in the team has the courage to tell other members that their ideas are rubbish. While many studios are now using some forms of metrics in order to get some idea of where the might be going right or wrong, at their core the decision-making often still tends toward groupthink rather edict. Right?

Wrong again. What happens is institutional cowardice, I'm-not-to-blame syndrome, and a project that actually devolves into a set of poorly-connected sub-projects and sub-teams who believe that at least "their bit" will be good even if they think the rest of the game stinks. Management by consensus might make everyone feel a bit better about themselves in the short term, but in the long term it results in massive amounts of cruft.

The two things that I would encourage any studio to start doing are:

1. Define design tests and metrics as a part of any work they do. You should state not just what your fancy mechanic is, but also how you propose to validate it with real people (not your testers, nor team members, nor the producer's son/wife/dog), see whether it's a good idea or not in reality and then - and only then - include it  as a part of the main project. Without that hard information, all you're doing is creating project bloat. 

2. Appoint a director whose principle job is to say "no" and to remove anyone from the team who can't work with that authority. Without that authority in place, the metrics are just going to lead to a lot of down-the-rabbit-hole arguments and get nothing done. Somebody needs to be in charge and call the play.