Kris Ligman is a news editor for Gamasutra.

With IndieCade's annual festival in Culver City well and truly wrapped up for the year, I thought I would take this opportunity to shed a bit of light on the selection process for the festival. As a two-time juror and one of this year's "super jurors," I've seen a side to the decision-making process many have not.

Whether you are a developer who has not yet applied or a former applicant who has come away confused about how IndieCade selections are chosen, I hope this can provide some insight into why IndieCade selects the games that it does.

Juror criteria

There is not much of a formal process to becoming a juror, though we're always looking for new ones. The best way to get involved is to get in touch with one of the existing jurors or organizers, or come out to one of our special juror sessions held over the summer in Los Angeles.

Jurors are invited based on their knowledge of games. This can mean experience in development, a background in game studies academia, work as a game critic or journalist, and the like. Some applicants might be a bit confused why IndieCade would bring in journalists and the like to participate in jurying, but the idea is to diversify evaluations as much as possible.

In my case, I've dabbled in all of those aforementioned areas: I curate Critical Distance, whose This Week in Video Game Criticism feature appears weekly here on Gamasutra; I studied games throughout college and graduate school; and prior to becoming Gamasutra's newest news editor, I served as a community lead for a kids' free-to-play game. So while I don't have the specialized area of expertise of a triple-A veteran, I'm sufficiently well-rounded that I can talk about games from a number of different angles.

This is useful for IndieCade, since we want to compare applicants to each other as little as possible, while also understanding each applicant in the context of its genre or cultural backdrop.

Also, it's worth mentioning that jurors are unpaid volunteers. Super jurors get a small stipend in exchange for the higher volume of work they perform, which I'll get to below.

Super jurors and first wave evaluations

Super jurors (yes, that is what we're called officially) were first mentioned at this year's Red Carpet Awards, where some of our feedback notes to our 36 finalists were read aloud on stage in the course of the ceremony. In truth, we had been at work all summer alongside regular jurors, as we actually participate as part of the first wave of evaluations all IndieCade submissions undergo.

IndieCade received some 800-odd submissions this year. Provided the files submitted work, those are then assigned to jurors and super jurors. Assuming that I'm anywhere near representative of an average, super jurors review more than twice the number of games assigned to other jurors.

Super jurors are chosen based on the depth and breadth of the feedback we've written to past applicants, on the assumption we'll continue to provide in-depth and potentially constructive responses on every new game we're assigned. In exchange for our efforts, our votes receive slightly extra weight than that of the general juror pool. A super juror's feedback can't determine on its own whether an entry will be accepted as a selection, but it can help the decision process if other jurors are split about a game.

In theory, every submission will be assigned to at least two jurors, although with an almost limitless amount of possible hardware and operating system configurations, it can be difficult to ensure every game receives its due consideration. We have a number of remedies for this, including asking applicants for setup instructions with their submissions and holding the special juror sessions I mentioned before, which has the benefit of herding a lot of jurors into the same room to interact with a game under identical specifications. This is doubly important for multiplayer and "big games" (board games, pervasive games, etc) which are nigh on impossible to set up for solo play.

After playing a game (ideally to completion, or at least long enough to get a good feel for the mechanics), jurors fill out review forms on our online system. I should note, in particular, that even though it's the most visible part of the feedback applicants receive from us, the written response section is just one component of our evaluations. Much of jurying is done through scoring on some undeniably subjective (yet still valuable) criteria, such as world design, innovation and even cultural significance -- things which are difficult to articulate and even harder to rank, and also don't necessarily add up to a "yes" or "no" vote.

From Ienién:te and the Peacemaker's Wampum.

An example: Ienién:te

One of the games I evaluated this year was Ienién:te and the Peacemaker's Wampum, a student game facilitated by Aboriginal Territories in Cyberspace (AbTeC). This is an organization that is all about the sharing of native experiences through new media like video games, something that is sorely underexplored in mainstream media.

Unfortunately, Ienién:te (pronounced yeh-YAWN-day) was not accepted into the festival. This was a shame for me personally, as I saw a ton of potential in the game -- the titular protagonist is sort of an anti-colonial Lara Croft, an archaeologist returning stolen artifacts to her people -- but if you play the game for yourself, you can see why it may not have worked so well in festival exhibition. It is a culturally significant game, for sure, but in a loud public space full of flash and spectacle, it wouldn't garner the attention it deserves. It is also (I'm sorry to say) a bit rough to play, and though polish isn't always a determining factor in IndieCade selections, you can see how in this case, many players would find it a resistant experience.

There are other examples I could name. I evaluated 43 submissions for this year's IndieCade, of which I would say I recommended about a quarter for either further consideration or inclusion at festival. Of those 43, 12 were selected for festival, and five of those 12 were shortlisted for awards. There are a number of games not among that five that I would have thought were just as worthy of Red Carpet Award consideration, but I don't want to turn this into a complaint session. I bring up Ienién:te just to illustrate that a game can be many compelling things -- original, hand-made, funny -- and still not make it past the first wave of evaluations, for a variety of factors.

That is why IndieCade looks for multiple jurors from diverse backgrounds to do these evaluations -- and also why jurors are just the first of multiple rounds of decision-making. A separate group is responsible for finalizing the selections based on juror recommendations; yet another group votes on the Red Carpet Awards. The Closing Awards are determined by community vote.

To be sure, it's not a perfect system. I'm not in a position to propose improvements, really, although I'm sure I speak on behalf of the organizers when I say we're always looking for more jurors. Inasmuch as evaluations of games can never be scientific, the more input a game receives the greater the likelihood that a concensus will emerge. At any rate, it will mean more and hopefully productive feedback for applicants as well.

What not to do

My advice is as subjective as any other super juror's and it is not the final say in any IndieCade decision. Nevertheless, all jurors do have a few guiding principles when we set out to evaluate submissions -- and a lot of it is common sense.

Don't send us your clones. No one likes a clone, even a pretty one. You would think that it wouldn't be quite so dire but I have seen enough physics platformers this year to last me well into my old age. I don't need a single one more. You should absolutely feel encouraged to work within an established genre if that is what you like to do, but please bring something of your own to it. One of this year's selections, Valdis Story: Abyssal City, is a great example of working within the Metroidvania mold while fleshing out an iconic art style and setting.

Seriously, don't send us your clones. I can't believe this needs to be repeated, but it does. And if you absolutely can't resist sending us your cloned or derivative game, for Sid Meier's sake don't mention what it's a derivative of in your application.

Don't send us an instruction manual that's thicker than the rest of your game materials combined. If you can't explain your game simply and elegantly to more-or-less trained professionals, it's not going to be pick-up-and-play material for regular Joe attendees and their youngsters.

It is fine that your game's narrative layer is thin, but don't make it superficial. Few jurors are going to be up for reading oodles of backstory, but your game's narrative layer should at least scale to the game you're trying to create. A dense introduction to explain why your MOBA units have different colors is not necessary and sets up the wrong expectations. Likewise, a lot of exploratory games could benefit from a bit of contextualization.

Don't send us a broken build. Somehow this worked out for Videoball, but in general I wouldn't recommend it.

Videoball, as seen not broken.

Comment is free

There's a lot I haven't addressed here that you should feel free to ask about in the comments. I can't speak for IndieCade outside of my capacity as a super juror, obviously, but inside of that role I am happy to answer questions or concerns about the evaluation process or elaborate on the feedback I provided.

Otherwise, if you're interested in getting involved as a juror yourself, I'd highly recommend it! Feel free to bother me any time on Twitter or contact the IndieCade organizers at [email protected].