League of Legends can't seem to shake its reputation for toxicity -- but it can make substantial changes to player behavior, as part of a ongoing campaign for change in the community.
League of Legends can't seem to shake its reputation for toxicity -- but it can make substantial changes to player behavior, as part of a ongoing campaign for change in the community. That's the message from Jeffrey Lin, the PhD psychologist who works at Riot Games as the company's lead designer of social systems.
This year at GDC he gave a talk on the game's player base and what the studio is doing to nudge it toward positive behavior, by-and-large. This interview, conducted during the show, drills down into the thinking behind its initiatives, and reveals a lot about what portion of the players are actually negative, how best to combat negativity, and concrete ideas about messaging and initiatives that can make a difference.
You've said that the toxic player-base is smaller than you might think. Can you tell me a little bit about that?
Jeffrey Lin: Yeah. So in our analysis of the whole player demographic, only 1 percent of players are the ones who are consistently homophobic, sexist, or racist. What's interesting, though, is that they're not responsible for a lot of the toxicity in the system.
So when you break down the ecosystem -- how much toxicity is sourced from that 1 percent -- it's only about 5 percent. Actually, the majority of toxicity is the neutral or positive players.
And the thing is that every once in a while, they'll have a bad day -- a bad day at work, bad day at school. They'll carry that into their game. 90 percent of the toxicity is those players. So you look at a 100-game history, and they may be only negative in three games. And the question for us is, "How do you solve that problem?"
But part two of that is what we learned this year: Even though a small minority is truly negative, they can control your community's perception by themselves. It just takes -- and this is really interesting -- 11 percent of negative posters on a forum discussion to just change the direction of the forum discussion.
You mean to derail it into negativity?
JL: Yep. Or what we saw, for example, was that the very first few upvotes or downvotes, that will actually change the eventual perception of the entire discussion -- just the first few.
How much of this is carrot-stick stuff? You want to change behavior. The Tribunal is "stick," I guess. How do you handle this?
JL: It's interesting you mention Tribunal, actually. In the past, a lot of people did use the stick first. Then the industry evolved a bit and we focused on, "Hey, maybe using the carrot is better." Frankly, the answer is we need both. A lot of people would say just use one or the other. There is no silver bullet to this kind of stuff.
So actually, looking backwards, it's about, "Who are the players who are affected by one or the other?" So the truly negative players, the 1 percent, they respond only to punishment -- the stick. The neutral players, they're the ones who respond to positive reinforcement. They're the ones you can sway a little bit more towards the right, a little more positive.
Looking back at Tribunal, which you mentioned, one of the new features we're doing is positive review. So it's not just about looking and reviewing negative behaviors, it's about positive behaviors, and what "good" looks like for players. So in the future, if you review someone who is positive, you get a small prize and they get a small prize as well. So we really want to do the full spectrum in League of Legends.
You said you want people to behave better. How do you influence them -- is it simply through things like rewards and positive reinforcement, or is there persuasion to people's higher reasoning?
JL: One of our guiding principles, and the focus of today's [GDC] talk, is we don't want to be the drivers. We don't want to be the arbitrators. We want the community itself to drive their own community. That's why we have something like Tribunal, where they're voting what's good or not good. Or what's okay or not okay. And actually, that's the key.
So when a player is in the game, and they have some negative behavior, they get a message saying that, "Hey, your peers don't think this is okay online. Your peers don't think this is cool." That's actually why we see the change.
And what we also see is that when you actually have the community drive everything, they get invested in it. And they feel like, "Hey, nobody is here telling us what is okay or not okay. We're defining it for ourselves, and we're making the right decisions." And they can see their own community drive over time. And that's always been the guiding principle since the very beginning.
When you ban someone high profile from the game for a period of time because of toxic behavior, do you have an understanding of what effect that has on the player-base, and what they think of that action, and how they perceive it?
JL: That was a really interesting case -- IWillDominate, right? A couple years ago, we made it a strict policy that we're not going to treat players differently because of status, or being a paying player. None of that matters to us. Every player is upheld to the same standards. In fact, in these days, we expect our pro players to have higher standards they need to adhere to.
What was interesting was that when we worked with IWillDominate, we needed to reset cultural norms. The problem was that players were logging into games, doing these kinds of things, and then thinking, "This is normal. It's okay here."
We needed to reset that. We needed to change it so that it was more like real life. Players were telling us things like, "Well, in real life, I'd never do something like that, but online it's different." We needed to fix that problem. And working with people like IWillDominate, like Ocelote -- Carlos.
Those players needed to be our prime examples of, "Hey, it's no longer cool to do this kind of stuff. It's no longer okay to be that way." That's how we actually started that approach to resetting cultural norms.
It's not just in League of Legends that you see this. There are other prominent examples in recent memory of people saying, "Online isn't real life, and this isn't how I behave in real life, and this isn't reality." But obviously it is.
JL: That's right.
So how do you recalibrate people's perception that online is reality?
JL: I totally agree with you. Both are the same. We're spending more and more of our time online. Our kids are growing up, and more and more of their time is spent online. Our expectations of both should be the same.
So what we did is, when we give players feedback on their behaviors -- and we have messages like, "Hey, your peers think this is not cool" -- they can actually share that back to the community.
So let's say you go on the forums and you complain: "Hey, I was banned by Riot. I don't agree. I don't deserve this." So now, other players are saying: "Hey, show us your evidence. Show us what they sent you." They'll post it, and all the other players will be like, "Hey, that's not cool. You should have known that." And that's how we're slowly changing that perception.
I was reading comments on one of the posts on your site, and someone said something like, "a certain amount of shit-talking should be expected in a competitive environment." I just want to know your take on that.
JL: I think that sentiment is okay. As a studio, we're not out to get rid of offensive language. We're not out to get rid of competitive banter. An interesting story there is, look at when the Tribunal first launched. When people were posting in the Tribunal and they saw offensive language, they'd post it all over social media.
They were like, "Riot, should we punish offensive language? What should we do here?" And we were like, "No, no, no. We want you guys to figure that out. Tell us what you think you guys should do." Over about six months, what you actually saw, the first couple of cases, it was about fifty-fifty. Half of the people were voting "punish" for offensive language, and half were not.
After six months, it settled. The community had agreed on something, and it was "as soon as you say offensive language and it's directed at somebody, that's verbal abuse, and that's not okay. That's never okay." But if it's just offensive language, like, "Oh, fuck. I missed that skill shot," nobody cares. So it's really cool to see that evolution over time in the community.
Something that pro sports has is standards for sportsmanlike conduct. I'm sure you've done research into it, and you well know this stuff.
JL: Yup.
Do you model any of your regulations on that? Because obviously, you're both pro, you're both sports, but there's also the huge cultural difference between the two.
JL: That's right.
So can you tell me about that?
JL: On the eSports side, another best person to talk about how they came up with their code of conduct is Dustin [Beck] or Whalen [Rozelle] would be better for that. But on the player side, we are actually trying something very new this year. In the old days, we had a Summoner's Code. It was kind of our code of conduct, so on and so forth.
But because we've learned so much about player feedback and how they can drive their own community, we're looking forward to trying something where the players can drive and vote on their own Summoner's Code, and we want that player code to replace our own Summoner's Code, and we're really looking forward to that this year.
How are they voting on it -- is it going to be that people suggest rules?
JL: We're not sure about the exact design just yet. But it is something like that. "Hey, what do you guys think are the worst behaviors that we as a community are against? What do you think are the upstanding behaviors that we represent," right? And we'll have people give us their suggestions, do their voting, and we'll see how it shakes up.
A lot of what you talked about is being guided by the players. But you have a PhD! You know your research; you know what you're talking about from the psychological perspective. Where does that meet the community, and what do you bring to the table that, say, a regular community manager who comes up through the community doesn't?
JL: Well, two things. At Riot Games, I was hired as a game designer, which is very different in the industry. Usually you see someone like me hired on the UX team, or the research team. But I was hired as a game designer. And the question was: "If a scientist who really understood this stuff had full reign to do this stuff on the game design side, could we make a better game?" And that was the guiding line in this.
If you look at something like matchmaking, how do we put players together in the same game? My classic example is something like, let's say you don't like offensive language. And I know that here's a bunch of players that never use offensive language. You would make a great team together. And nowadays, because we're game designers, we can influence the matchmaking of games. And innovate in that space as well.
Do you keep metadata on players? Because how do you know that I don't like offensive language?
JL: It's interesting. It's how you behave in the game. Let's say that after a game you saw something you didn't like, and you report that player. And now with all that metadata, we can now understand that, "Hey, you're against offensive language," or "you're into competitive banter." And based on that data we can change your experience to make one that suits you, customized to you.
Are there changes you can make to game design? This is wildly out there and I'm making this up, but say that, the atmosphere of the level or the way mechanics work pisses people off more? Is there anything of that nature?
JL: It's really interesting. We did a feature called "Team Builder" last year. One of the things about Team Builder is, in the early prototypes, we had this war music -- war drums. And it would ramp up over the course of the pre-game lobby; as more and more players joined, the music would swell. And it got to a point where it was, "you're charging into battle."
And what we saw in some studies was, actually it didn't promote cooperation. Because you have five strangers, you're in this lobby, the music is pounding -- and if there's a conflict, the music made it worse. It actually influenced the context around that situation.
So we do look at stuff like that -- the ambience of that. And nowadays what we're working on is a new version of Team Builder, you're coming in as strangers, we actually make the communication and cooperation super-easy for you. And the music is more toned-down, and it's only after you get over the tough decisions and the conflicting decisions that the music starts ramping back up again.
The thing I'm hearing is that is that it's not any one thing that affects this.
JL: That's right.
It can also be a bit subtle, or making the connection between what affects it is not as obvious.
JL: That's right. And in fact, it's as little as... A lot of developers come to me and say, "Isn't it as simple as adding real ID? If we make everybody use their real names, don't the player issues just go away?" It's not true. The little nuance there, is that if everybody has real names but there are no consequences, they won't change.
If you look at news websites today, it's almost exactly the same -- even though you post with your real name these days.
I have no idea what you're talking about!
JL: So with real ID, devs will say, "If they have to use their names to play, and everybody can see your real name, then all of the problems go away." But as you might have seen from news websites, a lot of them ask you to log in with Facebook, or log in with Google today, so they have your real name. But the behaviors haven't changed.
The problem there is because there are no consequences. Even if you're posting under your real name, you know that nothing is going to happen to you, so there's no reason to change. That's a small nuance. But somebody who has experience in the psychological fields will know, it's about consequences, not about the anonymity.
It can't all be negative consequences for bad behavior. Are positive consequences for good behavior as big a part of it?
JL: It's just as big. In League of Legends, we need to do more of it. If you break down League of Legends into positive players, neutral players, and toxic players -- really, really negative players -- negative, as I mentioned, are 1 percent. Neutral is the vast majority of our player-base, and I think it was about 10 percent that are positive.
Positive reinforcement is the best for the neutral players. But like I mentioned, it's the majority of your player-base. So most games today don't do things in the positive reinforcement side.
Last year, what we tested was, at the end of the year we said, "If you behave well in League of Legends you get a free mystery gift, a free skin." And that was one of our first experiments. From that data, what we saw was that we can do this a lot more. Because the benefits are amazing.
So what this year we're focusing on, what if through the entire year we keep surprising players? What if you never know when the next surprise is coming, you're eligible all the time for all the surprises? And that's a small nuance. We don't want to announce, "Be good in February, get a reward in March." We want to always use surprises.
These are adults, generally, playing the game. The majority of players are college and above. How do you do this without "happy kindergarten teacher telling you to be good"?
JL: I think the key goes back to the original principle -- we're not doing it. It's other players doing that. So even when you get feedback from the game, it says, "Hey, other players have voted that this behavior is not okay." Or they'll get a message, for example, that says, "Hey, usually, you're not like this. What's going on? Make sure you're more positive in your next game." So all of the message is relative to you, or the players around you. It's not about us doing that "kindergarten teacher" thing.
Is it very precise language that needs to be used in this instance, I'd think?
JL: If you don't use precise language, you'll still get effects. If you have precise language, you get double-digit benefits. When we say stuff like, "You usually are not like this, try to be more positive," we're actually linking that to the player: "We know that you're not a bad person. We know that you showed a bad behavior, but you're a good person at heart." That is usually enough to snap the neutral player, when they're having a bad day, back to the good path.
If you want more from Jeffrey Lin, you can watch his talk from last year's GDC -- free on Gamasutra.
You May Also Like
