Is Google AdWords an Effective Tool for Gathering Playtesters?

If you're familiar with Betteridge's law of headlines, you've already guessed the answer. If you're not, I won't keep you in suspense: the answer is almost certainly "No."

Prior to game development, I spent many years as a web development contractor and consultant. I have setup Google AdWord campaigns for several clients with good results-- lifetime sales that measurably exceeded advertising costs.

Since my game Lost Crypts entered open beta, I have been looking for sources of "tissue testers" (new users with no previous exposure to the game) beyond the friends and family circles. I've had good success with various game and game development forums such as Reddit's /r/gamedev "Feedback Friday" and the TIG Forums.

I've heard that advertising a web game through Google AdWords is a negative ROI proposition. The lifetime expected value of a player, even for a well-monetized game, is invariably less than the cost of acquiring that player.

But based on my previously positive experiences with Google Adwords, I decided to conduct an experiment: To see how effective Google AdWords is at bringing in players. Not with the goal of monetization (the game's monetization not being live at that point) but as a source of early playtesters and analytics data. One of the many nice things about AdWords is that this is a low-risk experiment. You can run a low volume, targeted campaign for a few dollars a day with no minimum buy.

How Google Adwords works: 

You create an ad campaign that serves one or more text ads. These can appear on Google search results or on various content pages in their network (including Google search results). Then you choose some search terms and optional demographic filters. 

Finally, you set a "maximum bid" for your ads. The maximum bid is the most you will pay when someone clicks on your ad. The bid is like an auction-- the highest bid will be shown in the best spots, the lower bids will be shown in less prominent spots. This can be set on a per-search term basis.

An ad campaign can run for as little money as you like, setting maximum bids for phrases or keywords and a daily dollar limit, although if an ad's click-through percentage drops too low, Google may automatically suspend it.

With an already active Google Adwords account, I spent an hour choosing a couple dozen search terms (such as "dungeon crawler" and "free web game") and writing several ads. For most of the search terms I set a max bid of $0.50, with a few set as high as $1.00. I limited the entire campaign to a maximum of $10/day.

The Results

As of the time of writing, the ads have been shown approximately 37,000 times, with 110 click-throughs (one visitor for every 340 views), at an average cost of 38 cents per click. So far the experiment has cost $41.

Unfortunately, the performance of those visitors has been poor. Out of those 110 visitors, only 52 triggered the event associated with the game starting to load. Here are some possible reasons for this initial drop off:

Okay, so $41 got me 52 players. Still not too expensive for playtesters, right?

The Funnel

Well, not quite. Starting the game load is just the first step to playing the game. While the game is very easy to get into, with no download or registration, the easy access nature of a web game makes it less sticky.

The pre-game flow:
Step 1 Game Load.
Step 2 Play as Guest or Login.
Step 3 Join a game.

First, according to my in-game analytics, the percentage of visitors who finish loading the game, enter a game instance and then play for 30 seconds or more is 50%.

But for Google Adwords that number is only 6%.

If we examine the early game funnel, it's not just that Google visitors are more likely to bounce during the loading screen, they drop out at faster rates through every step. Not one of the 52 visitors from Google Adwords ever finished the first level.

The funnel from landing to exiting the first level.

Here are the underlying numbers of players from all sources (sample size ~1000 unique players):

87% waited for the game to finish loading.

75% continued to the "Join Game" screen.

66% tried to join a game

50% played for more than 30 seconds

40% made it to the end of the first level.

Compare these numbers with just the players coming from Adwords clicks:

63% waited for the game to finish loading.

50% continued to the "Join Game" screen.

25% tried to join a game.

6% (4 players) played for more than 30 seconds.

0% made it to the end of the first level.

To double check this wasn't an issue with the analytics tool, I also looked at goal conversions using Google Analytics:

The two unblurred entries are the largest sources of Google AdWords traffic. Note their "game connect" conversions relative to other sources.

The Google sample size is admittedly too small to make specific conclusions, but spending $41 to acquire 4 players who were in the game for under two minutes is not a good deal. At this point, the expected cost of acquiring a player who plays for an extended period of time is unknown, but is potentially hundreds of dollars.

While I still think Google Adwords is a great system for some business cases, using it to acquire play testers is not likely to be cost effective.