Gamifying Reviews Won't Work

I read this piece on what Steam should do for indie developers and find myself in disagreement with a large number of points.

It’s hard to argue that Steam could and should be better than it is (the client has been showing signs of hanging together by a thread for a long while now as more and more stuff gets added to it) however much of the asks and recommendations in this piece, for me, seem to have not considered the consequences of what’s being asked for.

A very real case of ‘be careful what you wish for’ and, in some cases, a very real case of ‘I’m not entirely sure what you’re asking for’ as well. For now, I want to concentrate on the request to 'Make user reviews more useful' and to gamify them.

What Are User Reviews For?

‘Make user reviews more useful’ is kinda handwavey and vague. There’s a sort of explanation there but it doesn’t really make much sense and it comes across as a hodgepodge of handwaves and bouncing from point to point. The reviews aren't useful for me, so they're not useful for someone else and... something.

I'd argue that user reviews are incredibly helpful right now.

User reviews aren’t for developers, they’re user to user. That the majority of reviews on normal games treat them as product reviews is by design and desirable. This is the sort of quick fire information people want to know before laying down their cash.

Sometimes people just want to have a quick scan for a thumbs up to reassure themselves, a thing Metacritic played on to the detriment of videogames as a whole with its made up scores and weighting. Tallying up the percentages of customers leaving good or bad reviews may not be a perfect system but at least it’s not terrible like that.

Want criticism? Find critics.

If a developer wants in depth critique, deeper comments on how good their videogame is we already have systems in place for this. It’s called the press.

Of course I’ve been around long enough to know that getting press coverage for a small game is difficult, exponentially so in these times of an abundance of videogames. This is why I’ve long argued for developers to support smaller blogs and outlets as they go. Their traffic or audience might be small now but it’s a small investment in a possible future, where smaller writers either expand their audience or go to work at larger outlets. It’s about keeping a healthier eco system around games. It’s something that folks tend to put to one side whilst chasing new things that they can directly measure as causing a sales bump.

If you want thoughtful critique of your game, you seek out the critics. You go to where they are. They are, for the most part, not going to ply their trade or indulge you in the user reviews section on Steam any more than they are in the Amazon review section for Far Cry 27:It’s A Far Cry From Home. It’s not like Valve don’t cater to this either, there’s an explicit section in the store for reviews from elsewhere and, should you want it, an awards section too.

From personal experience on the developer side, I don’t have a great deal of reviews but that they’re positive acts as a small sales multiplier. It’s why developers desire the magic ‘overwhelmingly positive’ thumbs up. It helps. I’m very thankful to everyone who took the time out to write kind words and I appreciate those who were less than happy with my work too. They are already incredibly useful to me. Sure, I’d be sad if I’d gotten a ruck of bad reviews instead, heartbroken even, but that’s sort of how it goes.

The Press Is Not An Advert

A brief aside.

Oh for a quid for every time someone has said ‘the press doesn’t matter to indie games’ on the back of this sort of information without realising that it is not the job of the press to translate into direct sales, to act as an advert for them.

To date, even with the rise of video, thoughtful critique is mainly missing outside of the written word. Nothing yet has come along that replaces the multitude of things a good press can provide, not just for games but for the culture surrounding games. Show me a YouTube video as thoughtful as Donlan’s Night And The City and we’ll talk. Show me Bow Nigger on Twitch. Show me more than a handful of people and channels discussing games in an intimate and thoughtful manner and maybe we can begin to talk replacements. That’s not to say there aren’t a number of channels and people involved in more thoughtful stuff but right now, you couldn’t fit a weekly round up with it never mind on the regular.

Until we’re at a place where there’s at least some parity, let’s stop pretending that YouTube, Twitch or whatever other media of choice are a replacement for the press and not just media in addition to. Generally when you look at the arguments for this it’s people trying to measure a thing that’s not supposed to do the thing they’re measuring anyway so no wonder it turns up a blank. Press coverage will be difficult to map to sales in the way a YouTube feature or a streaming fest would be and it always will be. Press does have a substantial impact on sales for many titles, especially smaller ones (see last year's itch.io traffic to sales graph) but a great deal of its effect is indirect. Treating it as you would an advert and expecting results as if it were an advert is misguided.

And of course, there are those with a vested interest in playing up some sort of new media vs old media battle and yes, I'm sure it sounds smart in their head.

I find it all very depressing, y’know?

Rewarding The Wrong Things.

The suggested solution for all the handwavey problem of user reviews needing to somehow, for some reason, be more useful is to gamify the review process. I’m just not sure how this is supposed to work for anyone.

Gamifying reviews is not going to improve the quality or utility of them. It might increase the volume of them but in doing so it’s near guaranteed to also increase the noise and make them even less useful. All the stuff asking for leaderboards and whatever else is a complication where solutions already exist, even if those solutions themselves could be better. (A lack of press coverage for a lot of titles unfortunately boils down to a lack of resources on the press side to cover so many things. It’s tough out there in writing land too, we all feel the same squeezes in the end)

Worryingly, and sadly unsurprisingly in tech and videogames, it’s a request that also ignores the more negative sides of reviews. It should be clear to everyone by now that sometimes the internet gets more than a little bit of a bee in its bonnet over things, we’ve all seen it happen enough times.

It should be clear that sometimes the internet goes on a little crusade and sometimes, sometimes the internet stays on target to make things a better place but sometimes the internet decides a game should not exist for reasons unfathomable to most rational humans. It should be clear that review bombing can and is used to abuse small developers or developers found giving a shit about people by people who clearly give no shits because the internet is weird sometimes. Reviews are used as part of a pattern of abuse.

In gamifying the general review service, it would also be positively rewarding abuse. Suddenly folks aren’t just posting terrible things in order to sink a game, they’re getting rewarded for it too. As the complexity increases in the system, so does the workload for keeping the place clean. This is not desirable, nor should it be encouraged or welcomed. It is, however, the exact sort of thing we should be looking at avoiding when designing our systems but very rarely do so.

Without considering the long term effects of the changes we ask for, we risk throwing more people to the wolves, we risk offering more avenues for destructive elements in our communities to empower themselves.

We should be very careful about what we wish for in exchange for minor short term gains.