Steam has a lack of data scientists

--Intro
 

Before I say anything, I need to mention that I really like Steam and I want the platform to last longer whether or not I'm developing games for it. I also respect all the employees working for Steam and I think they're doing a great job. But they are outnumbered.

What happened?

On September 12, Steam made an update to its game discovery algorithm and changed how games were being shown to Steam users in several spots: 

-"More like this" section of "Other games pages"
-"Recommendations" on the home page
-Discovery queue (the weight of tags seems to decrease)

This was the latest change in discovery algorithm starting with:

- October 2018 update: Unreleased games started to appear less in Discovery queues 

- July 2019 update: Announcement of Steam Labs that introduced the new AI powered "Interactive Recommender" tool where the algorithm started to factor "sales" much more than other inputs. As a consequence, unreleased games -that don't have any sales-  don't show up at all.

In the release announcement of September 12 Updates, Steam told:

But it seems, the outcome is the exact opposite.

Until now, the "More like this" section and "Recommendations field", were relying on the tags applied to the game by the developer and the players. Depending on the similarity of their tags, games were being shown under each other. 

That old feature made possible for developers to "target" some games and use similar tags to gain some exposure. Now that it's gone, the new algorithm, that looks very similar to how "Interactive Recommender" works, is showing only popular released games of the main genres and if the genre is unclear, it's only showing popular games. 

This was the latest spot where smaller games were getting a little bit of exposure and now it's gone.

Here is how the daily wishlists look for our unreleased life sim game (Circadian City) that was being shown under 35 other games according to Steamlikes before the change:

Our daily average wishlist dropped from 75 to 10. We are not the only ones that have been hit hard by the new discovery algorithm.
 

The Kotaku article covered some of the complaints from developers. One developer tried to analyze the situation on his Twitter posts.

The rest of the developers I'm talking with are quite unhappy about the situation but they "grin and bear it" until a new change or a better store comes. 

We need to understand the Steam ecosystem in order to better analyze the situation. For this, I will break the ecosystem into smaller clusters and interpret the change from the perspective of each cluster:

--Players in Steam Ecosystem

1- Valve: 

As the owner of Steam, Valve wants to increase its profits and it also needs to protect the profitability of other players to save the ecosystem. Because without other players it can't do any profits.

2- Big publishers: 

Apart from Valve, there are some big publishers in the ecosystem that have the risk of running away from the store by either start dealing with Epic or use their own sales platforms. The latest changes seem to favor their games over smaller developers but I believe the increase in percentage in their sales can be ignored as very small of their page visits were coming from "More like this" and "recommendation feed" anyway.

3- Small developers: 

High quality indie titles also defines the quality of Steam. Yet, they are the biggest losers of the latest change. Most of them has been completely wiped out from the store.  Games like Don't Starve, Terraria, Stardew Valley has sold more than millions of copies and they are from small developers, after the latest changes they had a small boost to their visibility but the main problem here is that there is a much bigger polarization in terms of revenues for small developers.

Let's exclude mobile game ports, school projects and back catalog dumps to Steam that defines the "garbage" of the store and almost everyone is annoyed of. Apart from them, on Steam there are still thousands of high quality indie games developed in several years by talented developers that rely on the revenues of their games in order to survive. The rewards might be slightly bigger but the possibility to break-even in the current environment for a small indie developer/studio is much less than before. Even the most successful indie games everyone knows can't risk of start working on a  game knowing that it has a chance to succeed less than 1% even if the reward is higher than 100 times of the initial investment.

4- Light users:

According to the article Your Target Audience Doesn't Exist by Sergey Galyonkin from 2015 most Steam owners have only a few games in their account. This didn't change in the updated article by Sergey: Steam in 2017. There is a huge chunk of players that Steam wants to sell more games to and that's totally understandable. People playing PUBG or CS:GO might start buying more games and become more profitable for Steam. 

Some people argue that Steam made the latest changes to start milking these users but I don't think the changes will affect their behaviors at all. You can't turn them into power users whatever you do.

5- Power Users:

I consider myself as a power user with 355 games owned and over $4000 account value according to Steamdb. Let's consider everyone who purchased more than 100 games as a  power user. The number of them on Steam isn't that high, probably only 1-2% of total players on Steam are power users but they account for half of the total sales on Steam and furthermore, they are one of the most engaged, loyal customers I have ever seen in my life. 

As a power user, I can't find the games I want to buy or wishlist easily anymore. Almost all the games Steam is showing me are the games I either already own/wishlisted or I'm not interested. 

Furthermore, these power users do like quality indie games a lot but once it's not sustainable for indies to develop games for Steam anymore because of its risk and lack of exposure, the experience of these power users will deteriorate very quickly. 

Maybe Steam has an overconfidence for their loyalty and thinks they have nowhere to go with hundreds of games in their account. Abusing the loyalty of your customers is the biggest mistake a product can make, I hope Steam won't learn this the hard way.

Oh by the way, power users are also the reason everyone else is there. 

--The real problem

The number of light users increased a lot in recent years on Steam but I don't think Steam is trying to favor them over his engaged, loyal (power) users. 

Steam is trying to avoid any conflict of interests issues by removing all criteria that might seem subjective. I have no idea why they hold on to this approach but maybe they are afraid of one big publisher complaining that Steam is favoring games from Valve more than the other publishers. They don't want to deal with bad publicity because of their curation standards.

As a result, they didn't want to handpick games so they introduced Greenlight, then it seemed like Greenlight is being abused by bots then they introduced Steam Direct. 

It's the mentality of most Google products and we all know the state of mobile Google store compared to a more handpicked and curated App Store. 

In my opinion, Steam Direct was the biggest mistake Steam made and we're still having the repercussions of that mistake right now. 

Steam direct filled the store with games that no one plays. They created a competition for visibility spots and caused Steam to look for new measures for games discovery.

Furthermore, some publishers started to abuse the tags and wishlist system to gain advantage in visibility by dumping dozens of asset flip games. 

In this mess, Steam decided to rely on sales data rather than tags and wishlists. The Interactive Recommender and developer notes from August roundtables in Gamescom are supporting this argument. 

Now we are at the stage where all (almost all) upcoming games lost their exposure. We all know that wishlists are extremely important for a successful launch. They convert into sales with a rate of 0.2 to 0.5. Wishlists are good for developers and customers. Wishlists make purchasing habits less impulsive and gives the developer a chance to understand if their games is well expected or not so they can iterate on design or increase the budget of their games.

This new environment will be advantageous for games from big publishers with strong out of the store marketing and the survival rates of indies will drop significantly to the point where making indie games is not profitable anymore for anyone. 

Some people argue that, quality indie games will keep surviving but this is just a shortsighted thought process. Nobody knows if a game is going to be successful or not before the game is developed. The developers of FTL made their second game with a bigger development and marketing budget "Into the Breach" and it didn't sell half as much as their first title. The game is great by the way, but there is always a luck factor and it will be foolish to go forward without the support of a big publisher in the new meta. 

This will kick back the industry to where it was in the beginning of 2000. Steam was the pioneer of the indie game culture back then and now it seems it's destroying the thing it created.

 

--Suggestions for Steam

- Start thinking in clusters. Don't generalize everyone as players or developers. They have very different needs.

- Hire some data scientists. I'm sure they will create even more clusters than I quickly did in this post and they will start creating value for everyone and Steam.

- Stop showing me games that I already own/ wishlisted or have seen hundreds of times and not interested. I shouldn't have to ignore all the games in the store for the algorithm to understand I'm not interested in them. 

- Get rid of Steam Direct, start handpicking games for Steam.

- Don't abuse the loyalty of your power users. Don't choose quick profits over your engaged users if you can't create more personalized store experience for everyone.

-If you are making a decision, check it out if the outcomes are inline with your company values and the culture you created.

- Communicate better with developers. I'm developing games for Steam for the last 10 years and Gamescom was the first opportunity for me to meet a Steam representative face to face. It was a great start but it has to be much more. You call me partner in your contract but we barely know each other. Understand the needs of your developers.

- I'm not complaining for the 30% royalty but I believe I should get a better service in return of that. Give me some recommendations to improve my game, my marketing, maybe do some QA like consoles are doing or help me promote my game. 

- Hire more people. You are trying to run a huge business with a couple of people. From what I've seen Steam has great employees but they are unbelievably outnumbered to run a multi-billion dollar business. Hire developer relation experts, data analysts, data scientists, community managers, PR / marketing experts, project managers, developers. You have the resources, don't play that tight.

- Provide some attributed data. You show us the traffic breakdown for the store page which is great but you only show us the cumulative wishlist/sales reports. We don't know which traffic has better quality so we can focus our marketing efforts.

- Don't jump to conclusions from poor data interpretation. In your update announcement it was written

and you shared this image:
 

It's quite unfortunate that you only conducted this test for a few days and you only tested a single case. If you tried other variations you would see that the average visits are going up in every case.

When I changed my tags the last time and started to appear in "More like This" sections of some games, the click through rate of my game image was as high as 6% in the early days, than in a few weeks it converged to a dull 2% without changing the image. The reason is, the number of people checking on the more like this sections are limited and these sections are static for long periods of time. Whenever people see something new in there, they click on it. This means that your test for a few days doesn't show anything. Hire more data scientists and spend more time on your tests.

- Don't ruin your business, we all need Steam