Building a Recommender System for Destiny Players

by Kevin Zhai, Eric Pawlakos, Rohan Jha, Sai Narayanan Haran, Rafet Sifa, Diego Klabjan and Anders Drachen

Game Analytics represents a unique domain through which to apply machine learning. In the big data context, gaming data is high frequency and longitudinal. A game server may carefully track every single action of a player in their lifetime. All this data gives rise to a rich diversity of analyses from funnel analysis, churn prediction to engagement graphing and profiling. However, in Game Analytics, an area that seems to lack some attention is recommender systems (for a great introduction see here), i.e. systems that try to predict a useful further action, product or similar. In this post we report on our efforts to build a recommender system for Destiny trying to help players improve.

Recommender Systems in Games 

What is a recommender system exactly? If you use any kind of subscription service, you’ll be very familiar with it. At its core, a recommender system is a machine learning algorithm built to predict if a user will like a certain product. There are a variety of ways to implement these systems. Netflix uses collaborative filtering, looking only at what other users are doing similar to you, to recommend movies when you are chilling. Amazon utilizes content based approaches to recommend items, such as suggesting a certain fantasy book because you liked Lord of the Rings. 

We set out to build a hybrid recommender system for Destiny (the first Destiny, the work was performed just prior to Destiny 2 being launched). In this context, hybrid means a system that utilizes multiple approaches to build recommendations. In Destiny, players control a character that has access to hundreds of guns and special skills in order to defeat monsters and other players. For our analysis, we focused solely on the player vs. player (PvP) game mode. Fig. 1 shows the types of weapons that are preferred in PvP, giving a sense of the number of options a player has.

Recommender systems for gaming form a unique challenge. We are not talking about recommending which games to play; rather, recommending certain behaviors to increase performance in a game. MMOGs like Destiny are complex and require so much decision making that players often don’t even know how to improve. Should I use this skill tree or equip this weapon? The obvious answer to improve is to play more. Someone who has played 10,000 hours will be significantly better than someone who has played 10. Still, there must be a way to systematically help players improve through data. The thing that makes recommenders hard for gaming is that that different players will have different preferences over how they play a game. We wouldn’t want to suggest to a shotgun player that they should use a sniper rifle.

Figure 1: Distribution of Kills

Methodology

Data Preprocessing

The data was extracted through the Bungie API using a random sample of 10,000 players which provided comprehensive information about player behavior. This included PvP matches of the players, providing performance data such as number of kills with each weapon, average kill distance, deaths, etc. Additionally, the data covers the core character information about each of the players, giving us a look into what kind of stat allocations they used. There were two classes of stats that we cared about: battle stats and cooldown stats. Battle stats (agility, armor, recovery) affect the movement, health, and regeneration of a character. Cooldown stats (intellect, discipline, and strength) affect the cooldown of various special abilities. After dropping irrelevant columns, we converted some features into proportions (since number of games played or level discrepancies could skew the features) and standardized the data set.

Clustering

In order to make recommendations to players, we needed to find similar players to recommend against. Since playstyle is hard to quantify through one dimension, we developed 3 separate profiles through clustering for each character to capture the main pieces that factor into determining a player’s playstyle.

The first two profiles were created using k-means clustering on battle stats and cooldown stats, shown in Fig. 2 and 3. The heatmaps depict the feature means for each cluster. We chose 4 clusters for battle stats and 3 clusters for cooldown stats, based on interpretability and domain knowledge (many of the authors of this paper were avid Destiny players). We can see that players have different preferences for how they allocate these stats.

The third profile was created using archetypal analysis, shown in Fig. 4. Archetypal analysis is an unsupervised learning algorithm used to determine extreme entities, the archetypes, within the dataset. As such, we end up with unique profiles relative to centroid based methods like k-means. Most of the archetypes refer to players with specific weapon preferences, such as cluster 1 representing players who use auto rifles and 5 which are players who use shotguns. Other archetypes represent a general playstyle, such as 2 being a player who relies on timing their super ability to score massive amounts of points.

After this multi-profiling step, there were a total of 72 unique combinations (4 x 3 x 6). Every player falls into one of these combinations.

Figure 2: Battle Stat Clusters

Figure 3: Cooldown Stat Clusters

Figure 4: Playstyle archetypes

Recommender Framework

To make our recommendations, we consider the intersections of each profile, visually depicted in Fig. 5. Each intersection of the profiles represents the available pool of players that can then be recommended against. For ease of explanation, we use the scenario of Player X looking for a recommendation with our system.

Intersection 1 represents all of the players like Player X -- it is the pool of players who have the same combination of profiles as X. In other words, if Player X is in cluster 1 for battle stats, 2 for cooldown stats, and 5 for weapon playstyle, then intersection 1 for Player X is all players who are in those same profiles. Within that pool of players, we can then find players who have a higher KDA or higher combat rating (Bungie’s ranking system for PvP performance) and suggest to Player X to try out the weapons of those players. This approach allows us to use a collaborative filtering approach while still respecting the unique playstyle of the individual.

The benefit of our framework comes when Player X decides that they want recommendations that don’t consider players exactly like themselves. Let’s say that they want recommendations on cooldown stats. If we were to simply use intersection 1, the players we would consider would have the same distribution of cooldown stats as X, which would not be useful. Instead, we can look at intersection 2, where there is no restriction on the cooldown stats. Therefore, the players we now consider will have variability in cooldown stats, so we can show Player X others who are like them in playstyle and base stats, but varied in cooldown. The same process can be applied to other intersections.

In principle, our multi-profile framework can be generalized to n number of profiles, based on different behavioral aspects of any game. Such a framework provides a flexible way to incorporate different gameplay aspects and takes into account what players are willing to change.

Figure 5: Recommender Intersections

Evaluation on Reddit

To validate our recommender system, we had a couple of options. The best way to do so was to conduct longitudinal experimentation on a set of players and track if their combat rating actually improved due to our recommendations and not merely due to spending more time with the game. However, we had limited time and such a study would require a willing player population and more resources. We decided to do a qualitative evaluation to get some insight about the effectiveness of our results.

In order to survey players, first we needed to find them. Reddit was a natural source for this. The r/destinythegame subreddit is an active community of players that all play the game. We put out a post on the forum and asked players if they were interested in participating in a survey, offering a free copy of Destiny 2. After receiving about 50 responses, we collected in-game usernames and pulled their data directly from the public API. With the data, we ran our recommendation system on their data and created individualized reports, part of one shown in Fig. 6.

Figure 6: Final Individualized Recommendations

The feedback we received was very positive, with some reservations from a couple of players. Out of the players who responded, 80% of players found our recommendation helpful and would act on the recommendation. Most of the positive comments we received involved players enjoying learning both about other players like them and our suggestions on which guns to try. However, we did receive feedback stating that some of the recommendations were not very helpful since players all have different preferences. This brings us back to the problem of player personality -- some players simply want to get better through practice and not through weapon recommendations. Despite this, the main takeaway here is that players are hungry to know about which specific areas of the game they can improve on. Our framework represents one such way to do so, but it may only appeal to newcomers and not veterans of the game who know every detail already.

Conclusion

In our research, we set out to build a recommender system to help Destiny players improve at the game by suggesting different weapon loadouts and stat allocations. The key here is that we try to suggest actions that are still true to the player’s preferences. To do this, we first cluster the players based on their unique playstyle and stat preferences. Our hybrid recommender system involves taking different intersections of profiles and then recommending players that are similar, but with a higher combat rating. Our online evaluation on Reddit showed that the results were of great interest to players and that they would act on the recommendations if the system was a live service. Future work for our framework includes longitudinal experimentation to track metrics and validate if our players realize better performance. While our initial work was done on Destiny 1, a similar system is in the works for Destiny 2. 

Analytical systems designed to help players improve can have a huge impact on not only games like Destiny but also eSports in the future. The impact of data and analytics is pervasive in gaming, and as lifelong gamers, we are excited to see where it goes in the future.

For a longer version of this post (research paper) click here