'The Cloud' isn't as elastic as you think.

A few months ago, here on Gamasutra, I shared some thoughts on why I believe the world is not yet ready for streaming games. Since then, recent events have demonstrated that it may not just be streaming that our current infrastructure can’t handle. I had the opportunity last month to speak on Betanews about the challenges facing online content in “the Cloud,” and why consumer demand is outpacing the capacity of the cloud. With holiday gift-giving upon us, and a new wave of network-enabled console and PC gamers looming on the horizon, it seems more relevant than ever to share my thoughts about the elasticity of the cloud here on Gamasutra.

By now, we all can see that the Internet will become the dominant media distribution platform.

We can easily imagine (and expect) that all content will be digitized and distributed online. Every video game, along with every movie, every song, every software program, every live television event, every business document and more will be instantly searchable and retrievable via a range of connected devices. It will all be magically available to stream or download on-demand through "the Cloud."

This was the dream when I started my first Internet business in 1995. Now, people think it is the reality. And, while we are making great progress towards such a future, unfortunately, we are not there yet. The demand for online media consumption is simply too great and is growing even faster than we can deploy cloud services to meet it.

Because "the Cloud" is on-demand infrastructure that enables many businesses to scale without having to provision hardware, many people believe the Cloud is infinitely elastic. The cloud concept has become so familiar so quickly that it is commonplace to now hear "Just upload it to the Cloud" or "Just download it from the Cloud", as if it’s truly ubiquitous or that easy-to-use.

The Cloud may be elastic enough to meet certain needs, but it’s far from being ready to meet consumer media demand. Some examples during the month of November illustrate this point, such as the free-to-play release of DC Universe Online by Sony Online Entertainment. User numbers shot up at such a staggering rate that game servers were overwhelmed. The release of Call of Duty: Modern Warfare 3 caused the Activision gamer social network, Call of Duty Elite, to crash, and remain crippled for days. Even last month’s stream of the Republican candidates’ debate on CBSNews.com saw problems, where people were denied online access to the second half of the debate because of bandwidth limitations. The common theme of all of these situations is summed up by an Activision executive: "We knew there would be a tremendous amount of traffic. It was a lot more than we anticipated".

Media and game companies have not been the only ones bumping up against online distribution limitations. In July of this year, Apple planned to release the Lion operating system with only digital download distribution, but broadband limitations led to Apple offering two other methods: in-store digital downloads and an install via a USB flash drive. In January 2009, Microsoft had to delay the Windows 7 beta release because the anticipated download demand exceeded the capacity of the Microsoft servers.

All of these events experienced outages or degraded delivery. And this is not just a problem for one-time special events. Have you ever tried to stream Netflix in primetime? Can you imagine what will happen when millions of gamers download new releases and patches of popular MMOG game titles (some up to 16GBs in size) as the video game industry takes full advantage of digital distribution and embraces the free-to-play business model?

We speak of the cloud as being infinitely elastic but it is not. There are physical limitations. Surely, capacity will continue to improve, but many different solutions will be needed to help scale the cloud. More data centers filled with more hardware and using more bandwidth is not the only answer and certainly not the most efficient or green solution. Some solutions might include faster broadband networks, better compression technology, redesigned browsers optimized for media delivery, more efficient operating systems, smarter media players and client-side software that can make the cloud more elastic and allow capacity to scale up or down to match demand.

As more and more devices connect to the Internet, the demand for media delivery will continue to rise at an accelerated pace. I believe online media consumption will surpass industry projections, which were made to date in a certain context that is rapidly changing. We may look back at the early decades of the Internet and find there is, in fact, a Moore’s Law for the volume of online data delivered.