Big data and games - part 2 - algorithms and experts

In part 1 of the big data and games blog series, we described the characteristics of big data, and explained how big data would challenge the need for causation.  In this second installment, we will discuss how big data will affect the role of subject matter experts.  As before, we will draw on the experiences of Sonamine and the excellent book by Viktor Mayer-Schonberger and Kanneth Cukier.

Human subject matter experts?

Doctors are experts: they spend many years studying to gain a large amount of knowledge.  In their daily practice, they apply their knowledge to decisions in diagnoses and treatment, the outcome of which continue to hone their knowledge.  They have to keep up with new research and studies in order to refine their understanding of human body and diseases.

In their book,  Mayer-Schonberger and Cukier describe the work of Dr. Carolyn McGregor at the University of Ontario's Institute of Technology, studying the care of premature babies.  There, care of premees has taken on a big data like feel.  Patient data is captured in real time over 16 different streams such as heart rate, temperature and blood pressure, resulting in over 1260 data points per second per patient. 

Using big data algorithms, they have been able to predict the onset of infection 24 hours before any overt symptoms are observed.  And flying in the face of conventional treatment and wisdom, she found that very stable vital signs were observed prior to serious infection. 

Oh yes, Dr McGregor is a Ph.D in computer science, not a physician.

Human experts are critical in a small data world where people could not access enough or the right information. "In such a world, experience plays a critical role, since it is the long accumulation of latent knowledge - knowledge that one can't transmit easily or learn from a book, or perhaps even be consciously aware of - that enables one to make smarter decisions." (p.142)

What makes one expert "better" than another expert is usually the depth of their knowledge and experiences, and how often their decisions turn out "right".  Specialist doctors are considered experts in their fields; doctors with accurate diagnoses and good treatment.  It is but a small leap to suggest that more data = more knowledge = better expert. 

Big data and the new expert

Which leads to why big data sets the stage for expert systems... Where there is a lot of relevant data, both in the number of data points and the breadth of data types, that can be analyzed, big data enables a new type of expert - the non-human expert.  To take another view, a big data expert system essentially duplicates the learning and pattern recognition of a human expert, under circumstances that would totally swamp a human being.

Lest you think that big data cannot apply to creative endeavors, bear in mind that big data expert systems will work reasonably well when there is enough data points for the algorithms to glean patterns.  Consider the film industry, we now have over 30 million data points about commercial films going back decades.  The-Numbers.com crunches this data to provide predictions on the outcome of film projects.  Producers use this information in pitch meetings with investors. 

When a sufficiently large and accurate dataset of games is available, someone will apply big data methods to predict the outcome of game projects.

What does this mean for the game industry?

One management challenge here is evolve the game industry culture to embrace big data experts.  Unlike other scientifically oriented industries such as healthcare which are more open to data experts, the game industry historically has had an independent and creative streak.  It is not surprising that game industry veterans and experts decry the Zynga data driven approaches and recent corporate struggles, while secretly envying Zynga's financial success.  231M in revenues per quarter is nothing to sneeze at.

One approach is to divide and conquer: deploy big data expert systems where they shine and enhance human experts in other areas.

Management will have to decide where to use big data analysis or the human expert.  Since my company Sonamine has been leveraging big data in games, we have some experience that we can share on this topic:

- where there are more data points that are available than a human can comprehend, lean on the expert system.  These include predicting the behavior of millions of players, level progression, sales performance of virtual items and marketing campaigns, fraud, gold-mining.

- where you cannot access enough high quality data, lean on the human expert. A prime example is deciding where to spend advertising dollars. The highly fragmented user acquisition system with its many ecosystem players with siloed and non-correlated data makes it a challenge for any expert system.  Channels without tracking data such as TV ads make predictions difficult.  Simple heuristics and recent experience is what counts in a these situations. 

- where there is no data at all.  A good example is when a game designer wants to introduce a new game mechanic that does not exist.  Human judgement and small scale user testing will almost always win against a big data expert system.

Caveats and the future of experts in games

It would seem that the right thing to do would be to collect as much data as possible, since that would enable big data expert systems.  However there are downsides to that approach.  The cost and complexity to set up big data collection, storage and analysis capabilities sometimes outweigh the benefits.  I personally spent 10 years in the business intelligence industry where many companies wondered how to justify the ROI of expensive data warehouses and reporting systems.  In many cases, human experts will cost a fraction of that incurred with building a big data capability. 

I can already hear the outcry: big data will never create anything new; if we have looked at big data we would have gotten faster horses and not a car, and so on.  I agree, in fact there is another type of expert in medical science: these are the doctors that pioneer practices such as washing their hands before surgery. 

We have these experts in games too:  they are the creative game designers that invent a completely new mechanic.  They are the visionaries such as Jenova Chen, creating a game that tugs at the emotions of all players.  They are the marketers that decide to leverage live game play videos uploaded  to YouTube for viral acquisition.  They are the technical gurus that figure out how to enable players to blow up anything in Everquest Next. 

In my view, experts in the game industry must migrate towards being good at creating the new.  The new can be characterized by not much data being collected, unusual collaborations between different fields as well as just totally whacky stuff, such as the hyperloop. 

Secondly, algorithms can decipher the patterns; but only humans can make decisions.  And only humans can make decisions in a larger context of organization goals, objectives and targets.  As someone once told me, there must always be a fall person.  Blaming an algorithm for messing up is not going to fly. 

Conclusion

In summary, big data creates a new type of expert, one that crunches extremely large amounts of data to see new patterns.  The games industry can leverage these new expert systems to improve design, marketing and monetization. 

In part 3, the last of this series, we’ll discuss the concept of data value and apply it to games.

Reference

Mayer-Schonberger, V and Cukier K.  Big Data: A revolution that will transform how we live, work, and think.  Houghton Mifflin Harcourt, Boston, New York 2013 (Amazon link)