Yes, Let's Evolve the IGF

Cross-posted from my personal blog.

First things first: read Zach Gage's Evolving the IGF and then come back. He argues that the IGF could be highlighting more games in general, and more interesting-if-not-perfectly-executed games specifically, by changing the judging categories. This is an idea I wholeheartedly support, though I think there's room for discussion about what exactly the new categories should be.

I respond to his proposal directly at the bottom of this post. However, I've been wrestling with my own thoughts on the IGF and how it might be improved for a while now. I haven't said anything because I couldn't think of any good potential solutions to the problems I saw. With Gage's great idea in my face, however, I want to try to work through some of the thoughts that have been simmering on my brain's backburners.

My Angle of Approach

A bit of background regarding my experience with the IGF: I've never submitted a game to it, and this current IGF is the first one for which I've helped select nominees. This was not the first time I'd used a version of the IGF backend, however. I helped curate games for show at BitSummit, which uses a fork of the Indie Megabooth curation system, which is in turn a fork of the IGF system. At this point, I've used all three of those systems, and although there are a couple of differences between how IMB and BitSummit work when compared to IGF, the only significant differences are in how they want jurors to look at the games based on the goals of selection and the number of games to go through.

The order in which I've been a juror/curator/general looker-at-er of games is significant to how I think about the IGF and some of the complaints about it, so here's the order: BitSummit, IGF, Indie Megabooth.

It's significant because I first became aware of complaints about the IGF after helping curate for BitSummit and during the submission period for the IGF. I suggested that a developer friend of mine submit her game MidBoss for IGF consideration and she said that other devs had told her not to bother because it would be a waste of money. She'd heard that you have no chance of winning anything in the IGF unless your game is already well-known.

After a short pause, I had to admit that if the IGF worked very similarly to the BitSummit system (which was all I had seen at the time), that was an all-too-plausible problem. That conversation became one of the major reasons why I wanted to be an IGF juror. I wanted to see how similar the IGF system was to its grand-forkchild. And as I mentioned above, it's very similar.

Success Breeding Victory

Indeed, jurors do tend to gravitate to the most talked-about games of the year. Some people might be inclined to assume that jurors are taking advantage of the chance to play these games for free, and maybe some are. However, I think it's more that when a game like Undertale comes along and shakes up everyone's ideas about game design, it's taken for granted that the game is going to be a front runner. The kind of people who sign up to be jurors for the IGF are people who care about games as a medium and are going to want to have a say in how such an important game is seen. That's human nature at work.

I also think that a lot of the jurors had already played most of the games that made waves. I know that I had. There were over 700 games submitted to the IGF; even if I'd had every platform, there was no way I could play everything. So after playing through the games I was randomly assigned, the first thing I did was go through and look for stuff I'd already played, wanting to advocate for games that I loved but which haven't gotten a lot of attention and get in my two cents on games that I feel are overrated.

Looking up and rating games I was already familiar with was a way for me to get a word in on as many games as possible, as efficiently as possible. Human nature strikes again; you can only critically analyze so many games in one day, especially when you're also trying to get your own first game out the door and into the wild. By the end of the judging period, I'd rated about 10% of the total number of entries. About a third of that number were games I'd already played before IGF judging began. (The others were a mix of games randomly assigned to me, games that looked interesting to me from public comments made by other jurors, and games which I chose because they were clearly starving for attention.)

So yes, under the current system, games which have not gotten a lot of attention are at a disadvantage. Now, it's not completely impossible for unknown games to get nominations. Some of the jurors, like myself, go out of their way to chip away at the list of games which are being mostly ignored and leave public comments encouraging people to check them out if they stand out somehow. The system itself helps out with that task by providing filters to make that easier. Unfortunately, it's not enough to really address the problem. We still have a huge imbalance.

IGF vs. BitSummit/Indie Megabooth

The IGF is inherently very different from BitSummit or the Indie Megabooth and the way they approach game selection is also different. Although the selection process is mechanically more or less identical, the criteria by which selections are decided is different. The IGF seeks to highlight the "best" of its submissions, while BitSummit and Indie Megabooth are trying to curate an engaging event floor space which shows off not just quality but also variety. The latter two organizations also get far fewer submissions than does the IGF.

At present, assuming no modifications, I feel like the system works better for BitSummit and the Indie Megabooth than it does for the IGF. A simple game controlled by a shampoo bottle controller might be perfect for a show floor, but not have a prayer in hell of winning anything at the IGF even if jurors all received a custom controller to try it with. Whether such a game should have a chance at winning something at the IGF or not is up for debate, I suppose. However, I think we can all agree that if you're looking to create an interesting floor space full of attention-grabbing variety, it's easy to make room for weird little games like that.

I can't really back this feeling up with hard evidence, though. I've never attended the IGF in person; I haven't heard/seen devs complain about the Indie Megabooth selection process, but that doesn't mean there aren't any complaints floating around; and most of the Japanese developers who have attended BitSummit think locally, comparing BitSummit to Tokyo Game Show and the more recently established Tokyo Indie Fest. I could easily be missing a lot of information.

Let's Evolving

The reason I haven't talked about any of this before now is that I really didn't have any actionable suggestions for improving things. Everything I've thought of has obvious downsides and I felt like talking about the problems without offering any solutions wouldn't help anything.

Enter, then, Zach Gage's blog post and the category changes he proposes. I believe a massive shift in categories such as he proposes could improve on not just the problems he discusses regarding the design- and length-centrism of selection, but also make big steps towards getting lesser-known games more recognition. The keyword there, in relation to the lesser-known games problem, is could. I am not so sure that his proposed list of categories would significantly alleviate the problem of well-known games getting more attention and being more likely to win awards; it might just reshuffle things.

Granted, that doesn't seem to really be what Gage was going for when he made his list. He does skirt this idea a bit in his discussions of how often games are nominated in multiple categories, how that effects the total number of games nominated, and how jurors struggle with choosing games which are better designed over crazier titles with big ideas and poorer overall quality. But it doesn't seem to have been in the forefront of his mind. And to be fair, the issues he was definitely trying to address are good ones.

Anyway, let's look at the new category list he proposes:

I would like to state up front that I have a great deal of respect for this list and what it's trying to achieve. Very little overlap, so a greater number of games featured? Check. Subdivisions pitting games against each other by type of experience, rather than raw quality? Bravo. A separate category for multiplayer games, which deserve more recognition than they get now? Beautiful.

The best thing about this list he suggests is that it's tailor-made for games. After looking at Gage's proposed list, I feel like hindsight has punched me in the kidney. The current IGF category list now looks painfully tied to the award ceremonies used for film and theatre. Those award ceremonies undoubtedly influenced the IGF categories and were perhaps a good place to start, but film lengths and experiences don't vary as widely as those of video games do.

The worst thing about this list Gage suggests is that the length-based categories might be too broad. I see an elephant in the room, which is that so many indie games are procedurally generated, permadeath experiences meant to be played over and over again. Sometimes people spend dozens of hours playing games that are served in twenty-minute chunks. Where do those fit?

To use Crypt of the Necrodancer as an example: you can get a feel for the game in fifteen minutes and definitely find its "core", to use Gage's term from his Threes example, in under an hour (especially if you're already familiar with the roguelike genre). But to really see how deep its systems are, you have to learn more about the weapons and other equipment you can get, and thanks to the randomness of the procedural generation algorithms, that could easily take more than an hour. So what category might it go in?

I think Gage was looking for a solution which would have the same number of categories as the current format, and with that limitation in mind, this is a great list. I think it would be worth adding another category for games meant to be played over and over again to reduce confusion about how they fit into the length-based categories, though. That, or really buckle down on defining Long-, Short-, and Micro-form games and where infinitely replayable games fit in.

What About Lesser-Known Games?

To get back to the things I've been thinking about for the past couple of months: the reason I don't think this category shake up would help much with the problem of lesser-known games getting more traction comes back to the human nature thing. Jurors would still have played a lot of the things people were talking up already; they'd still comment on stuff they've already played before diving into the new stuff; etc.

The Micro-form, Long-form, and Multiplayer categories would definitely help increase the breadth of the nominees by giving types of games which usually get overshadowed a special place to shine. The lesser amount of overlap would mean a greater number of games featured.

I honestly can't predict how much of a difference shaking up the categories would make on this front. Can anyone? I do think it'd be worth a try. It's an idea that can be iterated upon, but if we don't try it, we'll never know.