Keep Playing with Me - Finding Retention Influencers in Destiny

This blogpost has been authored in collaboration with my colleagues (Anders Drachen and Johanna Pirker).

In social media we are getting used to see the #adv hashtag when an influencer is promoting a specific product or service. Influencers are people who leverage their connections with others to impact their behavior. Influencers have become a cornerstone of modern marketing strategies, where they encourage others to copy their choices and behaviors. However, influence is a much wider social phenomenon, and potentially any behavior can be influenced. 

In multi-player online games, the interaction between the players and the social community that forms in them play a fundamental role in the user experience and retention of players. Moreover, building and maintaining communities in games form an important aspect for the design and maintenance of persistent-world games.

In this post, we wanted to share our experiences from analysing the social networks for parts of the community of Destiny, specifically from team-based PVP matches. Destiny is a hybrid online shooter game that has attracted quite an impressive amount of players to compete or collaborate within a persistent online virtual environment. The “Crucible” is a major feature in the game which provides access to a variety of different PvP activities. Specifically, we were exploring if there are players in the community that have an outsized effect on the retention of other players. And if so, who these retention influencers were. 

The key takeaway from this post is that we did find users influencing other players’ level of activity within the game (e.g. how often and how frequently they played) to an outsized degree. However, they were far from being popular in the network of matches. Rather, influencers were involved in fewer but stronger connections. In addition, players in contact with those influencers tended to stay in the game for as long as the influencer played. Ergo, influencers could exert either a positive or negative retention influence, affecting others’ permanence in the game. 

In the below we provide a brief recap of the analysis performed on the PvP network in Destiny. For the complete details, please refer to our paper [1]. 

It is important to note that the term “influencer” is used about individuals or organisations that impart a behavioral or attitudinal effect on other parts of a social environment - in our case a network in the Destiny community. Influence can be positive, neutral or negative in terms of the impact that is desired. These terms do not indicate any moral or ethical values but are in effect mathematical variables in what is termed a social network analysis. 

Being Social in Games

Social Network Analysis (SNA) is the field of research and analysis dedicated to the study of social structures, using networks and graph theory. In game analytics, the popular shorthand mining the social graph is often used to encompass the broad array of analytical techniques that can be applied in SNA.

Social relationships can be modeled through a graph. A graph is a data structure that can be represented as a set of nodes connected together by binary edges. The nodes represent the users, which in this case are players, and the edges represent a social connection among two nodes. Modeling relationships through graphs allows much flexibility as, for instance, attributes (or features) can be associated to both nodes and edges.

Graphs, or networks, can be either static or dynamic. Static networks provide an aggregated view of the nodes’ properties and connections. On the other hand, a dynamic network preserves the temporal information. Dynamic networks can be built from a series of snapshots taken at specific points in time. In such a representation, edges can appear, disappear, or change their weights across snapshots. Similarly, if there are attributes associated with nodes, the values can vary over time.

Figure 1. An example of a dynamic graph in which the edges vary over time. They can increase their weight, appear or even disappear.

Social Network Analysis has been and still is widely studied in social media, which produces the most straightforward type of network. Research topics include, but are not limited to, the study of the topology of the network, the existence of emergent communities, and the presence of particularly important individuals (e.g., influencers). As SNA allows modeling social relationships, it is very much relevant also in games research and development. While this branch of research is still young, some researchers already took interest in analyzing player communities and relationships, mostly in communities built around games (e.g., on social media, or on third-party websites to create guilds). Very little (publicly available) research, however, has been conducted in identifying influencers within the game by analyzing telemetry data. Those are the influences that affect others’ behaviors through their in-game actions and interactions.

Here, we modeled PvP Crucible matches from Destiny. Thus, two nodes are connected if they participated in a match together as teammates. The edges also had a weight associated with them representing how many matched the two players shared.

Finding Influencers

Studies on social networks and communities, not only related to games, highlighted two main approaches to identify influencers. The first approach relies on the structure of the graph by studying users strategically positioned in the network. The second approach embraces the semantic nature of influencers, by considering the users that impact the behaviors of the individuals they get in contact with. In other terms, if other nodes are drawn to modify their attributes (or features) to resemble a specific node’s properties, such a node is an influencer.

Figure 2. Examples of the two approaches to identify influencers: the semantic and structural approach.

In the context of games, influence has little been studied. A notable exception is a work analyzing influencers in Tom Clancy’s: The Division [2], where influencers were identified through the structural approach - i.e., players well-positioned in the network were labeled as influencers. The work indicated that retention influencers, in the sense of these being members of the community that have an outsize impact on retention of other players - exist in games. Furthermore, the analysis from Tom Clancy’s: The Division showed that retention influencers are neither the elite players nor highly skilled players, who, to the contrary, appear to have little direct influence on the retention of others in their network. 

In the below, we expand on this study by comparing the outcomes of the two methods to identify retention influencers: the structural and the semantic approach.

Popularity vs Influence

To compute the influence of members of the Destiny network, we modeled the PvP Crucible matches through a graph. Then we applied the structural approach to identify central, or popular, players, and the semantic approach to identify influential players.

Popular players were the individuals strategically positioned in the network, retrieved as the nodes exhibiting high values in classical centrality measures on the network. Specifically, we measured the number of connections the node had (degree centrality), how accessible the node was to others (closeness centrality), the number of the shortest path in which the node was involved (betweenness centrality), the number of important connections (eigenvector centrality), and the portion of players accessible through direct links (pagerank). This process led to a sample of 51 central players.

Influential players were the individuals showing high values in the influence score metric. The influence score was designed to evaluate to what extent a node’s behavior was emulated by their neighborhood. In practice, the algorithm to compute the influence scores works in 2 phases:

The similarity is evaluated in the 1st phase, the computation of the edge influence.

There is similarity if after two nodes A and B get connected at time t-1, only one of the nodes (A) changes its properties at time t, while the other (B) is consistent with its past state. If this is the case, the magnitude of the similarity is computed using a similarity or distance function (which can be customized) upon the properties vector of A and B. B is the influencers, and thus the influence score is positive, while A is the influence, and thus the influence score is negative.

If either none of the nodes change their properties OR both the nodes change the value of their features the influence is not exerted. In the first case, there is no change. In the second case, either they came to an agreement or the change is casual - i.e., neither one of the nodes is more influential than the other in the specific connection. This process led to a sample of 100 influential users.

Our analysis led to two disjoint groups, 100 influential users and 51 central players. Influential users were far from being central in the network; rather they showed fewer stronger connections than central users. On the other hand, central players showed low values in the semantic influence, but a high variability in the edge influence scores. Thus, they affected their neighbors differently, as some were more influenced than others.

To evaluate whether, on the long term, influencers had an impact on others’ retention we used a custom metric: “retention transfer”.

While in the semantic influence algorithm we measured how players’ activity became similar once they interacted among each-other, retention transfer evaluates how the gameplay length of a node is similar to its’ neighbors’. In other words, whether the neighbors of the node analyzed left the game  when and if the node abandoned it. The retention transfer metric lies in the range of 0 and +infinite, where the smaller the value the more  permanence of the node in the game is emulated by its neighborhood. To compute the retention transfer of the node i, for each of its neighbors j connecting with i at a time t, we measured how close was the churn point, and thus, whether j left the game before or after i. In those two cases, the presence or absence of i in the game did not affect the node j, leading to a higher retention transfer value. On the other hand, when he more similar the gameplay length between the node and each of its neighbors, the retention transfer value is smaller.

Ideally, an influencer would affect their neighbors’ retention as they will stay in the game as long as the influencer is present. This derives from the definition of influencer as an individual whose behavior is emulated by others. It is important to note that emulating the influencers’ retention does not imply being retained for longer, as the influencer may decide to abandon the game before. In this case, we talk about negative influence.

Figure 3. The plots show the distribution of the retention transfer values among the (a) population, (b) central players, and (c) influential player

Comparing the retention transfer of the population, the central players and the influential players we found that influential players had a significantly lower retention transfer value. This suggests that influential nodes’ neighbors remained in the game as long as the influential individual was playing. In contrast, neighbors of central players were much less affected by the permanence in the game of those central individuals.

What’s Next?

Influence is a complex topic, as it is created across a broad range of effects involving the interactions between humans and their behaviors - and the degree to which these behaviors are visible to other humans. Social network analysis therefore cannot claim to understand everything that is going on in an online community. However, these techniques can be used to effectively and accurately identify players with different types of properties, for example influencers. If we are interested in working with players, being able to identify the players with the traits and behaviors of interest is vital. For instance, information on retention influence can be exploited to inform matchmaking algorithms and maximize the retention in the network of players for a specific game. We can also match players with retention influence characteristics with players who are at risk of leaving a game, for example due to not being able to progress through a particular point. 

In this post, we showed that retention influencers are not so obvious to identify in games and provided an initial approach to identify them. The existence of influencers opens the door to a series of new investigations and applications. For example, there are obvious implications for combating toxic behavior in employing SNA to inform matchmaking. 

The key takeaway: influencers impacting others’ retention do exist in online game communities and are far from being network-central, popular, or elite players. Rather, they are players involved in few and strong connections, also suggesting that influence may need to be reinforced over time.

References