Creating a Better Context for Competitive Play and Mastery: Critiquing Starcraft and League

I'm an avid competitive player in the higher levels of both Starcraft 2 and League of Legends. Both games have tried very hard to innovate in the world of Ranking Players. They've created entirely new and interesting systems that provide players with short and long term goals beyond the normal Elo number. However, I think these attempts to innovate are ultimately a failure, marred by obfuscation, confusion and negative contextualization. That said, I do believe in what Blizzard and Riot are trying to do and they deserve tons of credit for trying to solve an insanely hard design problem and trying to innovate in a field that almost no one tries to innovate in. So let's talk about what's right, what's wrong and where to keep moving forward.

How these systems work, briefly.

The League of Legends and Starcraft 2 ranking systems are very similar, so understanding one should give you a pretty good understanding of both systems. Both games split players into rankings by "medal" (Diamond, Platinum, Gold, Silver, Bronze). As you win, the game eventually puts you in a higher medal. If you lose a lot, you might be stuck at the bottom of your medal (League of Legends) or you might drop down to the lower medal (Starcraft 2). The method that the game uses to determine this is ostensibly points, which is a number next to your name that's supposed to represent your position in that medal. In reality, points are totally meaningless under the hood and the game barely uses them in its calculations at all. All the heavy lifting is done by a more standard but totally hidden Elo system that gives you a Elo rating called MMR and a rating of how confident it is that it's your real MMR. If you go on a massive winstreak or loss streak, the game's confidence in your MMR decreases and your MMR has wilder swings with each game. Your hidden MMR and your opponent's hidden MMR is what determines how many points you earn and therefore how quickly you climb or fall.  

(Image by Excalibur_Z from this excellent post) 

Both games also put you in small groups of players, where you can see up to 100 players who have the same medal and division as you. You're supposed to be interested in how these players are performing, but the game never actually gives you any real incentive to care. There's a number of nuances in both these systems that are relevant to our discussion, but too complicated to really explain thoroughly here. If you're interested, you can read this post on Starcraft 2 or this one on League of Legends. The biggest of these mechanics are Bonus Pool in Starcraft and Promotion Matches in League of Legends.

(A division of 100 Gold ranked 2v2 teams in Starcraft 2)

Obfuscation and Confusion

The most obvious problem with these systems is that they massively obfuscate and confuse your actual rating. Both games only display two things about you: what Medal/Division you're in and how many points in that Medal/Division you have. But those aren't the same things the game actually uses under the hood to calculate how good it thinks you are -- actually, they're not even close! Imagine two people playing Starcraft in Gold with 950 points. According to the information the game shows you, these players are basically exactly equal in skill right now. But Player 1 has been losing steadily and dropped from Platinum through the course of hundreds of games, so he has a very low MMR with a high confidence rating. Meanwhile. Player 2 is on a huge winstreak from Silver and she's only played 50-60 games total, meaning that she has a skyrocketing MMR with a very low confidence rating. Player 1 will gain 10 points from his next win and maybe lose more than 10 from his next loss, while Player 2 will gain 30 points from her next win and lose 3-5 from her next loss. Yet the game displays no difference in their ranking!

There are so many ways that this hidden MMR obfuscates and frustrates players. For one, your ranking and another player's ranking aren't really comparable, so it's hard to see where you stand except from a bird's eye view. It also seems like the game is arbitrarily giving or taking away points from you -- why'd I get 30 points for wins earlier but now I'm only getting 10? Why am I losing so many points when I was losing so few before?

Specifically, League of Legends has five divisions for each medal. This means that players must go through Gold V through Gold I in order to rank up to Platinum V. In order to advance to the next division (e.g. Gold V -> Gold IV), players must earn 100 points and then win 2 out of 3 games in an otherwise identical promotion series. For whatever reason, the game lets you soar through Gold V through Gold II easily, but once you reach Gold I and are trying to get to Platinum your points are completely irrelevant. That means that even if you have 99 points, if the game thinks your hidden MMR is below Platinum  it will start to give you 0 points for a win. So not only are points a bad indicator in both games, they actually become completely irrelevant if you're close to advancing a division. The only thing that matters is something the game is actively hiding from you.

Both systems fail massively in this way by obfuscating their true evaluations, and end up frustrating and confusing players.

Creating a Better Context for Competitive Play and Mastery

Where League and Starcraft's ranking systems really shine is in creating short and long term goals, which are really important for casual and competitive players alike. Presumably Riot and Blizzard decided they were willing to take the hit in frustration and confusion as long as they could earn back short and achievable goals for players. League of Legends' division system does a good job of providing small steps for anxious players, while Starcraft's goals are a bit more medium-term and spread out.

Although the short-term and long-term goals both companies create are really innovative and probably compel players to play more often than they otherwise would, they also create a larger and more subtle negative context for competitive play. This negative context is summed up in one phrase that players use all the time: "Climbing the ladder". Because everything in these ranking systems does point to a climb -- the short-term divisional goals and long-term medal climbing goals make earning points feel like a natural progression and losing points feel like a frustrating slide. Getting higher and higher in the divisional trees seems natural, while falling down divisions or medals will often cause a player to stop playing the game or become incredibly frustrated.

There's a lot about these short and long term goals that are really good at driving player motivation. But ultimately, I think they do players a disservice, because they motivate them the wrong way. They unintentionally tell players that mastery is linear, that the path to mastery is simple and clear and most importantly that because they play a lot they deserve to be masters. I think these negative contexts are responsible for some of the toxic behavior that players have when they start to fail and fall on these clearly outlined (but totally fake!) stepping stones.

This undermines the truth of mastery, the sublime experience of getting better at something. The truth is that mastery comes in strange ways, comes through study and in spurts or sometimes comes not at all for a very long time. These games are tying to force clean stepping stones onto a roaring river.

Traditional Elo systems do this a lot better, because they stay out of it entirely. They provide a ranking that lets the game and its players develop their own culture of mastery -- and you can see in games like Chess and Go that this culture might then become about study and practice and not about climbing and falling down. Getting better at Chess is not something you deserve, it's something you earn.

One way forward is a massive missed opportunity that sits attractively at the heart of both League and Starcraft's ranking systems: leveraging small virtual communities. Both games put you into a group of one hundred other players in an attempt to make play feel local and friendly. Bizarrely, they stop there and forfeit the possibility to make these communities something really compelling about playing competitively. What if these communities had chat rooms to talk to each other, and reasons to care about each others' ratings and play? What if there were intra and inter-divisional friendly competitions that encouraged strong players to help weaker players? There are a million ideas here, and I think they all have a lot of potential to transform the context of the way people play competitively. I'm confident that there are also completely new, undiscovered ways of giving players short-term goals without contextualizing it as a climb or as something players deserve.

I don't think we should turn our back on attempts to innovate, and again I think we have to give Blizzard and Riot as much respect as criticism -- they're innovating and trying new things on huge products when no one else is willing to. Is there a way to create a system that both has these motivational short-term goals and embraces practice and mastery as spiritual disciplines and not as frustrating, toxic climbs and falls? There's so much unexplored design space here, that I have to be hopeful and say emphatically YES, let's keep trying.