Analysing Auctions: The Case of Glitch

Please note: This post is a collaborative effort by Shawna Baskin (Blizzard), Joseph Riley (Pandora), Anders Drachen (GameAnalytics)  and Diego Klabjan (Northwestern University)

When scientists want to understand the specifics of genetics, they usually avoid studying humans, as it takes 30 years to reach a new generation. Instead they focus on fruit flies, which are born, mature and die on the same day. This allows for studying the causes and effects of changes occurring from one generation to the next.

The above is a loose rewording of The Innovators Dilemma, and it is a good description of the kind of rapid, iterative analysis that can be performed on populations of players of particular games. As churn rates in most games are exceptionally high, this allows analysts to examine and manipulate successive “generations” of users.

This is also the case for the graphically charming and endearing browser-based MMORPG Glitch. It survived for 14 months, but within that timespan it saw thousands of players successively enter and leave the game at varied rates, and exhibiting different types of behaviours while playing.

In this post we focus on just one aspect of Glitch: the in-game economic system, with an emphasis on the auction house and NPC vendors, which formed the nerve centre of the game’s economy. But first, a few words about the game itself.

The Game

Glitch: A Game of Giant Imagination

Glitch was a browser based MMO developed by a San Francisco based start up, Tiny Speck. It ran from November 14th, 2011 to December 9th, 2012, with the majority of its lifetime spent in beta. The core gameplay revolved around crafting, resource gathering, trading and social elements, all taking place in an open-ended world. The main objective was to build the world and create mini-games within.

Glitch operated with an in-game soft currency – Currants. Players could obtain the currency by questing, grinding/harvesting, or selling items to other players. Similar to other MMOs, players could post any quantity of an item in an auction house. Postings expired after 3 days, and Tiny Speck would claim a small fee for each of these items. Auctioning however, was not the only means of game transactions. Players could also trade privately with one another, completely bypassing the auction house.

The Data

We based the analysis of Glitch’s auction house activity on telemetry data that covers auction activity, other virtual economy activities, as well as general forum discussions and friend networks. This includes the key areas of auction sales data, item street prices, forum conversations, and in-game friendship networks. As in this post we will focus mainly on the way in which in-game parameters changed throughout the lifetime of the game, not all this data was included in the analysis presented here. Moreover, since friendship data and street prices changed little over the course of the game, we’ll leave them out of our discussion.

Over the course of 14 months, Glitch’s players performed approximately 3 million auctions. For every auction posted, we’ve collected the following data: player id, timestamp, action expiration date, item name, item category, item quantity, tool uses, tool capacity, and the final outcome of the auction (sold, expired, or deleted).

The Economy

At its peak, Glitch had 8357 Daily Active Users (DAU), viewed on a monthly average, number that gradually decreased to 67 DAU towards the game’s last month of existence. To better understand the game’s health in terms of player activity, we focused on studying the auction sales and forum postings, for both of which data was extracted from the game.

Distribution of Daily Activity

Auctions

Of the 3 million auctions carried out by the players, about 20,000 unique players listed 679 unique products across 41 unique categories. The number of in-game auctions dropped rapidly in the first month of the game, then stabilised until its final drop in the last month of the game.

Distribution of daily auction activity

The players that used the game’s auction functionality did so with great success: 85% of auction postings resulted in a sale. On a player level this figure changed slightly: there was a 35% success rate per player who posted in the auction. Despite this, the majority of players used the auction house rather infrequently.

The bulk of auction sales were for a small subset of items. 80% of all auction sales took place in the 10 main categories of products. Furthermore, 62% of auctions came from the top 5 categories, 28% of total auctions were for the top 10 products, and the most popular auction item was responsible for 8% of auctions.

Histogram of Item Auction Success as measured by proportion of auctions sold above the vendor price.

Street prices

In addition to the option of selling their items at the auction, players in Glitch could go to the vendor and receive 70% of the street price for their items in currants. Tiny Speck would periodically update these street prices. Overall, street prices skewed to the right, largely because of the outliers. For example, hooch (from the drink category), was the 3rd most popular item in the auctions, periodically sold for 1 million currants, although its regular price was 9 currants.

Auction Success Ratio Valuation for all Items sold (relative to whether the item was sold above street vendor prices).

40% of items depreciated in value, especially during the last 4 weeks of the game’s lifespan. This made us wonder how frequently an item would sell for less than 70% of its street value prices, i.e. the amount a vendor would pay for this item. In such a case, the auction becomes of no value to players. We called this proportion of this figure to sales a success rate.

From our analysis, we found that 59.1% of items would sell above street price 50% of the time, making auction more lucrative than other channels. However, many of the most frequently sold items, such as meat, butterfly milk and hooch, had success rates below 50%. Meat, for example, sold above vendor price only 25% of the time. These results highlight the importance of monitoring and regulating sales channels in MMORPGs.

Glitch artwork: Tiny Speck recently released all art assets from the game to the public domain.

The Behaviour

We also looked into variations in player behaviour, in terms of their use of the auction house and other economic factors, over the entire lifetime of the game. Using monthly bins combined with clustering (an unsupervised method in machine learning), we identified four consistent high-level clusters of behaviors in the player community: casual players, moderate players, forum posters and hardcore players.

Players did not remain in one single cluster over the duration of their lifetime with Glitch. For example, players who were active in Glitch’s economy for over 7 months (half of the game’s lifespan), moved between 4.4 clusters in their lifetime.

We will write more about the clusters and how players migrated between them in a future post, but in brief, each category exhibited a unique set of behaviours. Two of the most interesting categories were the unduly titled casual players and hardcore players:

Casual Players
Players were most likely to enter and leave the game as a casual player. Looking at the distribution of casual players in any given month showed a lack of players near the end-game months. We can speculate that players first get accustomed to the auction system as a casual player, and as their interests in the game fades, they return to a casual status.

Hardcore Players
In contrast to the casual players, hardcore players are more likely to remain hardcore players in the preceding and proceeding months. In fact, about 50% of hardcore players did so. Hardcore players are also the least likely to leave the game. Especially in the early month of data tracking, hardcore auction users and forum posters were most likely to stick with the game until the end. To qualify this a bit more, let’s briefly take a look at the numbers. Hardcore players remained in the game for 5.2 months on average, forum posters for 7.4 months, while casual users only stayed for 3.2 months.

The observation in the difference between the casual and hardcore players in Glitch leads to the well-known requirement for monitoring user engagement, which underpins any mature analytics practice. By inference, it also means that it is important to have the tools to track engagement metrics, e.g. to be able to identify whether a specific player is moving from a high-engagement, low-churn rate profile to a low-engagement, high-churn rate profile, so preemptive action can be taken to prevent player departure.

Conclusions

To summarise, a few of the conclusions made based on Glitch’s economy include:

In the next post on Glitch, due out in a few days, we will present a technique for dynamically analyzing and visualizing player behavior as a function of time. Stay tuned. 

Cross-posted from the Game Analytics research blog. The investigation described and the post was written by Shawna Baskin, Joseph Riley, Diego Klabjan and Anders Drachen. Thanks to Peter Landwehr at Carnegie Mellon University for making the Glitch dataset available for analysis.