Originally from Fortress of Doors, follow my stuff here:

Over two years ago I wrote an article called The Holy Grail of Digital Distribution where I described what an ideal digital game store might look like. In the last two weeks, Steam launched their long-anticipated storefront overhaul (which I'll call "NewSteam"), and I think it's time to assess how close they got.

The "magical dream store" I envisioned should:

"Merit" is a subjective term to be sure. I define it here as "games that have a potential audience, if only they could find it." When a store rewards games based on "merit" in this sense, it's connecting its audience with stuff they really want, rather than artificially feeding them stuff that's popular because its already popular, or stuff that bought its way to the top of the charts.

My proposal was to merge the best features of Steam with the discovery engine of the popular web game portal Kongregate.

Let's see how they did:

Discovery is the category with the biggest wins -- We got everything I asked for. We even got stuff I didn't ask for like:

Which are all big wins. The most important thing is to move away from Apple's totally broken top chart systems.

Submissions aren't totally open yet, but I'm almost certain greenlight will be replaced with something very much like Apple and Kongregate's open submission policies.

Out of everything I asked for, the only part that gets a mixed review is the handling of new content. The classic problem of totally open platforms is separating the signal from the noise. However, in filtering away the noise you have to be careful lest you also filter out weird, experimental, and/or niche titles that don't have mass appeal but nevertheless deserve a chance to shine. (I talked about ways to overcome this problem in another article, An Algorithm for Discovering "Hidden Gems".)

A big concern people have is the wholesale replacement for "New Releases" with "Popular New Releases." Although we saw increases in organic traffic and sales since the launch of NewSteam, we're also worried about how new games are supposed to fare. It used to be launch was one of the most lucrative periods for your game, but now it's the most vulnerable -- you don't get a free spot on the front page, and you don't yet have the benefit of an established reputation to drive organic recommendations.

Kongregate and NewGrounds deal with this issue with their "Under Judgment" systems, a common feature seen on user-generated content sites. Slashdot calls it the "Firehose" and on Reddit it's the "New" tab -- a clearly marked place for the newest, freshest content, with no promises whatsoever about quality. This keeps unproven content off of the front page but still guarantees it a minimum amount of exposure. And in the best implementations, it withholds judgment on any kind of final score until it gets a critical mass of votes -- a small but crucial defense against the "bandwagon effect."

So all in all I give NewSteam a B+. Now, let's talk about what effects it's been having and make some predictions of where it's going.

Our Results

Defender's Quest is an old game, nearly three years at this point, but has enjoyed a very nice long tail -- or Stegosaurus Tail, accounting for the spikes! Even so, it had trended nearly all the way down to zero daily sales as more games launched on Steam and increased competition for attention.

We can't share detailed stats, but sales are definitely and significantly up since the launch of NewSteam. According to the fancy new traffic analysis Steam has provided us with (and explicitly allows us to share), the vast majority of that is coming from the new organic discovery algorithm:

I've culled that list down to show only the top sources of traffic. We don't have any data from before the launch of NewSteam, but based on our sales I can take a wild guess that it would have been zero in nearly all of these categories.

Here's the really crazy part:

Our sales are up even though we're in the middle of a Humble Bundle!!

Defender's Quest is currently featured in the Humble Weekly Bundle: Leading Ladies, which started two days after the launch of NewSteam. Not only have our Steam sales not gone straight to zero, they've stayed at the level we saw right after NewSteam launched. We've posted a prominent announcement on our Steam page advertising the bundle so that people can get the best deal, and even still, a significant number of people are buying the game at full price.

The last time we were featured in a Weekly Humble Bundle, our steam sales went straight to zero -- probably because any meager traffic we were generating was coming from our own channels, and those customers would immediately know about a Humble Bundle.

So my only conclusion is that Steam's new organic recommendation engine is so strong that it is attracting an entirely new stream of players from the store itself, the kind of people who don't read my blog, follow me or my friends on Twitter, etc. If you're one of those people and somehow made it all the way here, grab the bundle and save yourself some bucks while you still can!

Curators

Traffic from steam curators is included in our traffic analysis reports, but it was so low that I removed it from the earlier image as well as dozens of other meager sources of traffic. In all, curators provided 0.6% of our page visits.

To put that in context, Defender's Quest is moderately popular with Curators -- we're featured by the Newgrounds Curator, who currently clocks in at spot #29 with 4,115 followers. (By contrast, TotalBiscuit, who enjoys spot #1, currently has 222,385).

If you're lucky enough to have a nod from a top curator you'll probably see more traffic:

(Robert's game Cthulhu Saves the World was curated by TotalBiscuit)

It's still largely unknown what effect curators will really have on the system, but I'll go ahead and make a quantifiable prediction based on the fact that Steam has 100 million accounts but TotalBiscuit "only" has 1.7 million subscribers on YouTube:

Let's see how well I do!

Down with Metacritic

Steam has started favoring the placement of their own user rating system over Metacritic scores. I'm pleased to see they modeled the aggregate score after the Tomatometer, and I can only hope they remembered How Not To Sort By Average Rating.

The aggregate steam user review score is now what appears on google rather than Metacritic:

This is a game-changer. Despite having sold over 230,000 copies in its lifetime and being widely critically-acclaimed, Defender's Quest still does not have an official metacritic score. This is because of the sites that happened to review us during our launch, only two were officially "approved" by MetaCritic. Historically, MetaCritic scores have been prominently displayed alongside games, especially when shown in list form (such as on a "featured Mac games" or "games under $10" list). Even though I hate MetaCritic, whenever I see that a game is missing that little colored dot I can't help but think, "gee, that game probably sucks" -- even though our game is in that exact same situation! I can't quantify it, but I'm sure this has reduced our potential sales.

In NewSteam, a score description now shows up on your game's store page, along with helpful mouseover text:

I'm not entirely sure what goes into calculating the final assessment, but I can identify several metrics that Steam's algorithm might be considering for any given review:

It still seems possible for a dedicated "bandwagon" or "hate-brigade" to ride into town and artificially distort a game's reviews, but this is a step in the right direction, and having multiple levers to tweak is a good thing.

One thing I've noticed is that nearly everything on my shiny new "recommended for you" Steam front page has an aggregate user rating of 88% or above, with most things well above 90%. Those last 10 percentage points seem to be a nearly logarithmic curve -- so far I've seen tons of 90% ratings, lots of 95% ratings, only one or two 99%'s, and no 100%'s.

Clearly, user ratings are a big part of the new algorithmic recommendation engine, which means we should expect to see attempts to game the system. Let's hope it stands up to the assault.

Fatter tails, harder launches

We were one of the last games to experience the "Golden Age" of Steam launches. There were only about 1,000 games on Steam back then, so you could easily stay on the New Releases tab for a week or more and bask in the exposure. After you rotated off the front page, sales would dip, but since there was so little clutter on the store, people would easily stumble across your game just by browing around.

Things changed in the greenlight era -- not only were indie games being added by the boatload, publishers were dumping their back catalogs and developers of all shapes and sizes were launching early access titles. We started to see a lot of backlash and the rising specter of buyer's remorse. I worried that an endless parade of sales and an impulse-buying feeding frenzy might be filling players' backlogs with stuff they didn't even want in the first place.

So what have we got now?

Launches will still be important, but I don't think we'll see the same kind of crazy launch spikes we used to -- assuming you were the kind of person lucky enough to pull one of those off in the first place.

The Good News

In general, NewSteam seems like a net positive, and I predict:

The game has shifted slightly from securing promotion to making the sale. Now that games are being fed a small but steady diet of targeted traffic, it's all about projecting quality when interested parties show up at your door. This means that games with an untapped audience will likely be rewarded over time. Conversely, absolute stinkers will suffer -- there's no longer a free pass where you get lots of traffic before people realize they've been duped.

However, this does not mean that all games that perform poorly under the new system are necessarily bad.

The Bad News

I think games that will have a particularly hard time with NewSteam are games that:

None of which marks them as "bad" in my book. These are simply an unfortunate combination of factors that will likely get your steam review score trashed, whether your game deserves it or not.

Moving forward

Since the publication of my Stegosaurus Tail article, I've started using the verb "To Thagomize" to describe the act of promoting a game's sales long after its release -- taking its name from "Thagomizer," the scientific name for the spikey bit of a Stegosaurus tail.

The art of Thagomizing used to be all about getting real friendly with your distributors and trying to chase one promotion after another. In the brave new world of NewSteam, Thagomizing is going to be just as much about keeping Steggy's tail nice and fat as it is about growing tall, pointy, spikes.

I for one cautiously welcome our NewSteam overlords.