Author: by Duncan Tebbs

In this second installment of a three-part series, seasoned console developers, formerly of Criterion, describe the process of moving to HTML5 development, covering the essentials, including timers and graphics. The first part of the series can be found here.

In Part 1 of this series, we introduced HTML5 for developing high quality games, giving an overview of what developers can expect when transitioning from other platforms. In this second article we talk about specific areas of functionality provided by HTML5 and related technologies, with particular focus on games.

In some cases there are multiple interfaces for a given subsystem, each with their own characteristics and availability. We discuss how these interfaces can be used to deliver a high quality gaming experience across a range of browsers and platforms, and talk about strategies we have adopted in implementing the Turbulenz game platform.

An example of the level of quality that can be achieved with well-optimized code using the latest browser APIs can be seen in this presentation of the Turbulenz Engine at WebGL Camp Europe.

Core Functionality

The JavaScript runtime environment has a collection of default global objects that are always present and mostly standardized across browsers. Here we describe those with a particular relevance to games. w3schools is a very popular and practical place to look for detailed JavaScript documentation but the Mozilla Developer Network is often regarded as more accurate and prescribing better practices.

Window

The window object represents the page that the code is running in. Browsers also define this object as the global object for the JavaScript context, and it becomes the implicit this when calling functions directly.

This global window object contains all other global objects, meaning that these two statements behave identically:

var r = window.Math.sqrt(4); // explicit
var r = Math.sqrt(4); // implicit

We recommend using window explicitly when checking for the existence of global objects.

The window object provides some fundamental functionality to the application:

Math

The Math object provides scalar mathematical operations and constants -- possibly one of the most reliable features across browsers.

All mathematical operations and constants provide double precision floating point values. Basic math operations like addition also work with double precision if the arguments do not fit on a 32-bit signed integer.

JavaScript does not provide SIMD support or any kind of vector and matrix library. Turbulenz has its own vector math library optimized to reduce object allocation and memory usage by supporting a destination parameter on every operation and by using typed arrays. The physics demos, included in our SDK samples give a good idea of the level of performance that can be achieved in optimized JavaScript.

Date

The Date constructor creates Date objects that provide basic date and time functionality. This constructor also provides a useful method called now that returns the number of milliseconds since the Unix epoch, without the need to create a Date object. The resolution of the time information varies by browser and is generally between 4 milliseconds to 15 milliseconds.

XMLHttpRequest

This object provides support for making HTTP requests. These requests could be asking for a specific resource or may represent operations to execute on the server. Browsers use the HTTP protocol as the main form of communication between the client and the server.

Although the object XMLHttpRequest requires different creation code depending on the browser (generally Internet Explorer is different from the rest) it has become common functionality so only really ancient browsers will lack support for it. Code like the following can be used to create an XMLHTTPRequest:

var xhr;

if (window.XMLHttpRequest)

{

xhr = new window.XMLHttpRequest();

}

else if (window.ActiveXObject)

{

xhr = new window.ActiveXObject("Microsoft.XMLHTTP");

}

else

{

return false;

}

Despite its name, XMLHttpRequest can download any resource type, text or binary. This code demonstrates how to support loading binary data:

xhr.open("GET", url, true);

if (xhr.hasOwnProperty && xhr.hasOwnProperty("responseType"))

{

xhr.responseType = "arraybuffer";

}

else if (xhr.overrideMimeType)

{

xhr.overrideMimeType("text/plain; charset=x-user-defined");

}

else

{

xhr.setRequestHeader("Content-Type", "text/plain; charset=x-user-defined");

}

xhr.send(null);

You can then access the resource data:

var buffer;

if (xhr.responseType === "arraybuffer")

{

buffer = xhr.response;

}

else if (xhr.mozResponseArrayBuffer)

{

buffer = xhr.mozResponseArrayBuffer;

}

else

{

var text = xhr.responseText;

var numChars = text.length;

buffer = [];

buffer.length = numChars;

for (var i = 0; i < numChars; i += 1)

{

buffer[i] = (text.charCodeAt(i) & 0xff);

}

}

We will talk about resource loading in more detail in a later article.

Extended Functionality

Several APIs required for modern games became available for developers under the umbrella of HTML5. Not all of them are actually part of the HTML5 standard but the term is often used to refer to all of them. Not all browsers support all of the APIs, in some cases making it impossible to execute a game that has certain quality requirements. For example, if your game requires advanced 3D graphics then you will require support for the WebGL API.

Due to the diversity of the browsers and the reluctance of some browser vendors to support specific APIs, all the browsers lack at least one or two APIs. Many developers will have to fallback on alternative solutions to achieve the required quality. The only fallback option that provides for all quality needs is a browser plugin, which must be explicitly installed by the user.

A large number of users do not like installing plugins for their browsers. In the past, badly implemented plugins would frequently crash the browser and use up huge amounts of CPU and memory. Plugins could also expose security threats, allowing access and control of the user’s computer by external parties without the user’s consent.

Of course every piece of software potentially has these issues, but the fact that some plugins did not update frequently enough and that external developers could not inspect their source code did not help to improve their real or perceived threat.

Modern browsers usually run plugins in a separate process in order to alleviate some of the security and stability problems listed above. If the plugin crashes it only takes down its own process, not the whole browser, and if there are security issues they only compromise the limited resources of the plugin process.

If the plugin process is using too much CPU it will only stall its own process and the browser can continue to respond. Browsers also now routinely check for updated versions of plugins and warm the user of required updates. Some also blacklist plugins known to be problematic.

So, although the situation has improved a great deal, users will still think twice before installing plugins, and many users will simply refuse to install them.

The approach of the Turbulenz engine has several aspects to it:

We consider our plugin strategy as transitional. As support for HTML5 and related APIs improves and we no longer need the features provided by our plugin our low level libraries will use the browser functionality. Note that this requires no change to game code and no action on the part of end users.

To learn more about the Turbulenz Engine technology see the SDK documentation, publicly available at http://docs.turbulenz.com/.

Asset Formats - JSON

The JSON object provides support for JavaScript Object Notation, which is useful for serializing data to and from strings. We recommended it as the best way to generate JavaScript objects at runtime from serialized data.

Modern browsers implement JSON support with optimized C++ code so in many cases it will outperform any other serialization method, particularly those that are not natively supported by the browser and require implementation in JavaScript.

Old browsers do not have JSON support, and in this case emulation in JavaScript is available.

Timers

The traditional functions setTimeout and setInterval only had a resolution of 10-15ms and didn’t provide any context information for the browsers. The first issue meant that smooth animations at traditional 30FPS or 60FPS were almost impossible. The second issue made it hard for the browser to guess the purpose of a given callback, and whether or not the code could gracefully handle being called less often (for example when the tab went to the background or became hidden). Modern browsers have increased the resolution of these traditional functions to about 4ms, but this is still not enough of an improvement for games. Thankfully, there are now alternative APIs for timing and animation.

requestAnimationFrame

The requestAnimationFrame API provides a more optimal and accurate callback for updating dynamic content in an attempt to fix the issues with setTimeout and setInterval. Browsers will try to call the callback at 60FPS, dropping to a lower frequency if the page goes to the background. Some browsers accept an optional parameter that specifies the HTML element to be updated by the callback. In this way the browser can optimize the re-composition of the page.

The API behaves like the original setTimeout in that you have to keep calling it if the callback is to be invoked repeatedly. The API took a while to standardize across browsers so code should try all the different names the function was given during its conception:

var requestAnimationFrame = (window.requestAnimationFrame ||

window.mozRequestAnimationFrame ||

window.webkitRequestAnimationFrame ||

window.oRequestAnimationFrame ||

window.msRequestAnimationFrame);

if (requestAnimationFrame)

{

var update = function updateFn()

{

// Perform rendering update

...

// Request next callback

requestAnimationFrame(update, canvas);

};

// Request initial callback

requestAnimationFrame(update, canvas);

}

High Resolution Time

The High Resolution Time API provides the current time in sub-millisecond resolution, more suitable for accurate timing than the Date functionality.

Most modern browsers support this API, although still prefixed with vendor names until the standard is ratified.

When the Turbulenz libraries fall back to the plugin for other functionality, they can take advantage of our implementation of setTimeout and setInterval with sub-millisecond accuracy to provide a solid 60FPS framerate. The issue of giving time back to the browser still needs addressing, and the plugin handles this by disabling rendering when the page goes to the background.

Graphics

Canvas

The canvas element provides access to procedural rendering APIs, as opposed to the declarative support offered by HTML itself.

For many people canvas means the 2D context API that was originally provided, although nowadays developers can also request a 3D API that we will describe later on. The 2D context API provides similar functionality to that provided by the vectorized graphics standard for SVG images.

Once a reference is available to a canvas element in the DOM, a 2D rendering context can be requested:

ctx = canvas.getContext('2d');

ctx.save();

ctx.translate(100, 100);

for (var i = 0; i < 6; i += 1)

{

var red = Math.floor(255 - (42.5 * i));

for (var j = 0; j < 6; j += 1)

{

var green = Math.floor(255 - (42.5 * j));

ctx.fillStyle = 'rgb(' + red + ',' + green + ',0)';

ctx.fillRect(j * 25, i * 25, 25, 25);

}

}

ctx.restore();

Although very useful, we have noticed several issues with the 2D API:

If a game only requires sprites then the 2D context API will work fine, but for complex vector graphics much better performance can be achieved with the 3D API which, for example, allows baking shapes into vertex and index buffers for optimal reuse. The lack of complex custom operations per pixel also limits what can be achieved with the 2D context. In contrast the 3D API provides support for pixel programs.

Old browsers use software rendering for the 2D operations, which will not perform well for complex scenes. Also, not all old implementations actually supported the standard. Modern browsers do provide hardware acceleration, and canvas support is now the norm.

The size of a canvas element does not change automatically in the same way other HTML elements can. The developer must manually change the width and height of canvas element to react to changes in the dimensions of its parent HTML element for example.

At Turbulenz we have implemented an emulation of the 2D context API on top of our low level 3D rendering API. This emulation cuts some corners for performance reasons so it will fail strict compliance tests, but it is useful for mixing 2D graphics with 3D rendering using a familiar API. This emulation on top of the abstracted 3D layer also helps in situations when either the browser does not provide canvas support (in which case our implementation will fall back to our plugin) or in the case when the browser’s implementation does not use hardware accelerated rendering (because our implementation uses the hardware accelerated WebGL API).

WebGL

The WebGL specification defines an API very similar to OpenGL ES 2.0 that operates on a canvas element.

It is not possible to obtain a 2D context and a 3D context from the same canvas element. Developers must choose beforehand which to use.

WebGL has evolved over a considerable period of time and so code of this form may be required to fully check for support:

function getAvailableContext(canvas, params)

{

if (canvas.getContext)

{

var contexts = ['webgl', 'experimental-webgl'];

var numContexts = contexts.length;

for (var i = 0; i < numContexts; i += 1)

{

try

{

var ctx = canvas.getContext(contexts[i], params);

if (ctx)

{

return ctx;

}

}

catch (ex)

{

}

}

}

return null;

}

var canvasParams = {

alpha: false,

stencil: true,

antialias: false

};

var gl = getAvailableContext(canvas, canvasParams);

One of the goals of WebGL was to provide a 3D API supporting a variety of devices: from high end desktops to mobile phones. OpenGL ES 2.0 matched that requirement with millions of mobile phones now being shipped with hardware support for the standard. This minimum common denominator handicaps modern desktop video cards by imposing an API that is at least a couple of generations behind those available on modern PCs. Extensions can and will improve support for more high end features but currently progress in this area is slow.

Some features not available with WebGL but commonly supported by desktop video cards include:

The WebGL shader model also lags behind what modern video cards support.

Support for the WebGL API needs backing from more browsers; not all of them support it and some only support it on certain platforms. Every platform has its own set of issues. On Windows the quality of some OpenGL drivers has led to the creation of a project called ANGLE to emulate the OpenGL ES API using Direct3D, which usually has more stable drivers. ANGLE has progressed massively since its beginning, but at the time of writing, the translation from an OpenGL-like API to Direct3D affects performance negatively.

Some browsers switch between using OpenGL directly or using ANGLE depending on how much they trust the drivers for the video card, which can create inconsistent behavior between machines or even the same machine when updating the drivers (these are inconsistencies on top of those that already exist in the hardware drivers themselves). Some browsers gave up on OpenGL altogether when running on Windows and always use ANGLE. Some browsers support WebGL on some platforms and not on others. We found that when using ANGLE shader compilation and linking usually takes a long time, often stalling rendering for several frames. We found similar issues when uploading custom texture data to the GPU.

Browsers support a very limited number of image file formats that they can load into textures. The “safe” set includes only PNG and JPG, PNG being the only one with support for an alpha channel. Any other image format will have limited support. For simple file formats it is possible to decode the pixel data using JavaScript after loading the binary data with XMLHttpRequest.

At Turbulenz we designed a low level 3D API at a slightly higher level than WebGL, similar in concept to Direct3D and based on shaders and materials. This API allows us to switch between different backends depending on browser support, and if the browser does not support WebGL we use our own plugin that natively implements our low level 3D API.

The games available to play at turbulenz.com, as well as this tech demo of the Turbulenz Engine show the high level of fidelity that can be achieved using the APIs described here.

Fullscreen

One traditional limitation of browsers when trying to provide a gaming experience similar to native desktop games is the lack of a fullscreen mode. The Fullscreen API was created to address this, and works by allowing JavaScript code to request a specific HTML element that will take over the whole screen. Fullscreen mode can provide increased performance for canvas rendering, allowing the browsers to avoid full page composition of the canvas contents, presenting just your rendering output to the screen.

The fullscreen API requires that the requests for going full screen originate from a user action, such as clicking a button or pressing a specific key. Code cannot just request fullscreen at will. This requirement originates from the need for browsers to avoid malign code taking control of the screen without the user’s consent.

This API still needs wider adoption as not all modern browsers support it.

Conclusion

In this second article we have described several features of HTML5 and other standards that developers have at their disposal for games for the web. In the final article we continue this discussion for the remaining areas of game development (Audio, Input, Threading, etc), and also discuss issues such as resource loading and security.