Analysis: what's with Steam refunds and short games?

[The GameDiscoverCo game discovery newsletter is written by ‘how people find your game’ expert & GameDiscoverCo founder Simon Carless, and is a regular look at how people discover and buy video games in the 2020s.]

So, you may have seen a viral post last week from the Russian developer of Summer of ‘58, Emika Games, who posted on Twitter with the following message:

As you might expect, this set off a plethora of news posts about the issue, because it tied in to a couple of trends that people already had in their heads: ‘short games don’t do well on Steam’, and ‘people take advantage of 2-hour refunds on Steam games’. And you can certainly run the story swiftly as ‘yep, the dev said this, it’s bad!’

Just looking at Summer of ‘58 for a second, Toge Productions’ ‘reviews by length of time played’ tracker does indeed show the majority of reviewers had played the game for less than 2 hours (though not many of the reviewers had refunded):

But what was Summer Of ‘58’s actual refund rate? When researching this article, I even noticed a video from the UK’s Independent website which says “the majority of those who played indie game Summer of 58 refunded it after completing it”.

We tried reaching out to the dev on Twitter to ask about his precise refund rate, but he never replied. However, somebody on social media spotted that he posted in Russian language that his refund rate was “something like 30%” across all of his games. So that’s the closest we’ll get for now.

And I’m not doubting that the game or his portfolio may have a significant refund rate. Just to set a baseline here, we did an article on refund rates last year, and said that for a portfolio we had access to, “average monthly refund rate by unit across the portfolio ranged between 5% and 8%. Actual dollar refund rate varied between 6.5% and 11%. (Refunds from higher-priced countries skew the dollar refund rate upwards.)”

But as we noted: “just looking at raw units, the highest recent refund rate for a game [in this portfolio] during a 1-month period was around 17%, and the lowest was around 3%.” So I think that’s the first point - refund rates can change a lot, and at their highest can be towards 20% or even more.

We do feel like sentiment moved swiftly towards ‘Steam sucks, because all short games get refunded a lot’, though. And from what we can tell, this isn’t true. For example, David Syzmanski (A Wolf In Autumn) commented “my experience releasing short games (which was admittedly years ago now) was that my refund rates were still lower than what we see even with games like Dusk.”

And Eric Juvi added: “I released a short game [Stars Die] on Steam on December 2020 and refunds are almost non-existent. The system could be abused, but players choose not to. Not sure what's going on with that game in particular.” We’re also aware of multiple popular recent ‘shorter game’ releases with refunds within the percentages we referenced last year (so, single digits).

So what, short games don’t get refunded more? It’s also not quite as easy as that - Crossy Road creator Andy Sum posted some Steam refund messages from his short horror game The Watson-Scott Test, noting: “it costs $5 full price, and the description says it’s 20 minutes [long].”

This would be a situation where refunds would rocket up, right? We asked Andy what his Steam refund rate was, and he told us it was 10.7% by number of units. This is definitely somewhat above average - but nothing like the numbers many might expect, given those refund messages.

What accounts for the large discrepancies in refund rates for Emika Games’ titles, and what can/should we do about it? One possibility: if your game is a single-play title which finishes within the refund window, the feeling you leave players with can be very important.

And if you look at a playthrough of Summer of ‘58 (walkthrough below, please buy the game if you want to play it!) you’ll discover it ends very abruptly, after 65 minutes, with a ‘TO BE CONTINUED’ graphic. So for those tempted to refund, you’re definitely given a giant ‘this is done’ signal - without a clear end to the story:

So sure, you can concentrate on the feeling you give players at the end of a short game. (Though there’s an argument that, if you don’t artistically want to give players a sense of finality, why should you be penalized?)

I do also wonder whether 3D jumpscare horror games that are shown to a super-wide YouTube audience - which Summer of ‘58 clearly is - attracts younger, more casual players. They may be more apt to refund due to maturity - or simply cashflow - reasons.

So we shouldn’t doubt the dev’s statements. But just noting that even at 25-30% refunds, you’re still getting 70-75% of the revenue. So I don’t know that refunds alone are normally a reason to quit making games. Honestly, Summer of ‘58 was doing pretty well in terms of reviews and had a lot of streamers picking it up?

And… the game is doing even better now! Saying how badly you are doing can sometimes spike sales. For example, Brigador picked up a lot of interest after being public about their issues. Similar things are happening here for this game, which was - after all - a well-crafted jumpscare fest.

Concluding: most short games don’t have the apparent sky-high refund rates of Summer of ‘58. But we suspect very short (<90 minute) games have somewhat above-average refund rates in aggregate - which is compounding what is already a tough commercial pitch. (But: it’s the ‘people don’t generally want to buy short games’ issue which is a much bigger issue than the refund issue?)

And we do think the ability to refund is a good, consumer-friendly thing for Steam to do. But Steam refunds are used by certain sectors of the audience to get their money back in circumstances that wouldn’t be acceptable to most other ‘you pay money for something’ situations. And that’s not great. So… it’s complicated?

[We’re GameDiscoverCo, a new agency based around one simple issue: how do players find, buy and enjoy your premium PC or console game? You can subscribe to GameDiscoverCo Plus to get access to exclusive newsletters, eBook and a Discord, plus interactive daily rankings of every unreleased Steam game, and more besides.]