Platform agnostic HTML5 game development and performance

Not long after Goodboy’s birth; one of our fathers, John Denton, wrote this post discussing the potential for native-quality mobile games running in browser. Two years on, what does this platform-agnostic promised land we dared to dream of look like?

If you’re short on time, I’ll cut to the chase and say it’s looking pretty cool! We’ve certainly been pleased as punch to play our part in shaping it with Pixi.js. Additionally, we have felt the demand for high-quality browser games steadily increase worldwide; giving us the opportunity to work with some major players like CBBC, Disney, PBS and Razorfish... making some award winning games together!

But is HTML5 capable of branching out into our living rooms? To sit alongside the Master Chief, Mario and Sonic, providing gameplay snacks and treats whilst that latest mega patch for your favourite game is downloading and installing? (The one that “absolutely will, we promise, fix all of the game breaking bugs that have been there since Christmas... this time... for real”)

Before we get too emotional, let’s be clear, packaged downloads and installs are here to stay for quite a while, but it has been thrilling to see others out there following and building the browser trail with us.  I’m not just talking about Tech Evangelists on Twitter... in the last two years, if they hadn’t already, all the major players and platform holders in videogames have opened up their hardware to HTML5 in a big way.

Back in 2013, it was only really viable to talk about Pixi.js based entertainment running on mobile and desktop browsers, here we are in Marty Mcfly’s future 2015 about to talk about real, breathing, games consoles as well!

What better way to get a feel for the lay of this new land, than a video of Goodboy’s games, running silky smooth on Xbox One!? Roll VT!

Originally we planned a fanboy-baiting comparison video of Xbox vs PS4 vs WiiU, but (SPOILERS!) while the games do run on the missing consoles, the gap in performance and frame rate was too big to make the video of value.

If you’re hungry for more, we are offering a full technical breakdown and comparison of tasty performance stats for dessert! (Service is not included in the bill, but we’ll happily accept some friendly tweets)

Our Experiments:

The Touchy-feely emotional test:
The most important test we could conduct, of course, was the good old fashioned gameplay test. We played 3 Goodboy HTML5 games on each device to see how they performed!

None of these were originally designed to take advantage of any console hardware such as gamepads or gyroscopes; being developed with mobile and desktop devices in mind. So it was fascinating to see how well they performed and controlled.

On Xbox One:

On PS4:

Results were not so hot, suggesting the stock webkit browser is not tuned for games, despite a very positive html5test score. None of the three games had sound playback, whilst Grubrunner was the only game that had a smooth framerate. It was noted however, that when tested a few months back All at Breakfast had a much smoother framerate, so the PS4 may have had an update that has reduced this, or gets up to things in the background that can hurt performance.

On Wii U:

Whilst performance was markedly choppy, it was a pretty cool find to see that Spice Dash detected the Gamepads gyroscope as an input, even if it was inversed (tilt Right to go Left). Nintendo has a pretty cool tool to demonstrate gamepad detection here so, if their browser performance improves, it could be a cool platform to investigate accelerometer and gyroscope use.

The New 3DS:

Whilst it's HTML5test score is high, it fell very short when it came to running all games. Frame rates were in the single digits with no sound, so it isn’t really a viable platform right now. However, it does support Stereoscopic 3D HTML5 video, so there is scope there for some creative web work.

PSvita was tested for us by Gustavo Parrado, and was found to have very similar performance to Wii U, since it's browser is also NetFrontNX based. If you’d like to contribute test scores, video or anecdotal discoveries about our games on any interesting device that we haven’t tested, please tweet us.

The Bunnymark test:

Since the dawn of time, man has oft-appreciated the humongous insight that throwing a large number of bunnies at something can give. So for our next, non-leporiphobic friendly, but slightly more scientific test, we booted up Bunnymark v3 in the Goodboy labs and took a look at how many bunnies we could literally fill each device with before the frame rate no longer held stable.To be clear, for animal safety, as soon as it dipped and flickered to 59fps or below and didn't go back to a steady 60, we pulled those bunnies out of there, counted them and wrote that number down in our copybooks. Those fluffy, energetic, little guys revealed the following results:

Unfortunately, this test proved a little unreliable and due to some technical issues, we weren’t able to get scores for half of the machines on test. We will likely come back and update this data down the road when we update our bunnytest tool. Do let us know what you get!

Going on intuition, however, the comparative bar lengths of the various machines we did test don’t seem far off what one would expect though!

The Construct 2 sbperftest:

Stepping outside of Pixi, Construct 2 is a popular cross-platform HTML5 based game engine. They have a cool little automated performance test: sbperftest that automatically measures the average framerate throughout the time it’s running to give you a good idea of how well a game will run. Basically, the closer a device gets to a score of 60, the better it is:

The New 3DS failed the test, whilst Wii U had an average framerate of 6fps.By comparison Xbox One clocked in just under desktop and mobile with a score of 57fps, sadly, PS4’s average was 8fps.Thanks again to Gustavo Parrado for contributing the test score for PSVita, which matched Wii U with 6fps. Scores for 360 and PS3 will be added in the future.

The HTML5Test test:

This final test does what it says on the tin, giving your browser a mark out of 555 for how many of the features and standards it supports. We navigated to this test on each device and jotted down the score, as well as any additional notes regarding supported features that weren’t discovered elsewhere.

No surprise to see Google blazing ahead with Chrome here, but pretty interesting to see how much support has advanced between console generations (PS3-PS4, 360-Xbox One) as well as how high the New 3DS and PS4 mark, despite their slow game performance. Perhaps an indicator that browser games weren’t on Nintendo and Sony’s mind, whereas maybe they are with Microsoft and its 8-player Gamepad API integration?

Hardware Feature support:

During the course of all these tests and wider research, a lot of interesting specs were unearthed regarding gameplay enhancing hardware and software features that were or weren’t available on each platform. Here they are, displayed for your perusal and future reference.

...I should probably deal with the elephant in the room now, as much as we all wish for it (almost as much as for Shenmue 3), Sega is yet to add HTML5 support to the Dreamcast, the universally accepted best console in the world ever, so you won’t find it in the chart above  :*( .

The Future:

So, right now, if you are looking to make browser delivered HTML5 games for the console audiences, your lead platform would have to be Xbox One, they'll run fast, support 8 gamepads and can be pinned to the dahsboard. Though games aren’t smooth, on Ps4 and Wii U, they are playable and do show some future potential.

One topic which always comes up when discussing HTML5 games is monetisation, or the limited options therein, but the platform holders do all have options for you as a HTML5 games developer, which could be worth keeping an eye on!

On Apple’s App Store (and Google Play), you can sell your game as a packaged app, either as a webview or a compiled hybrid and therefore monetise in the same way as a native app. You just need to make a good game and get it noticed!

Developers can build Apps for Xbox One using HTML and Javascript.

And Nintendo actively courts the Indie development scene, allowing self publishing of HTML5 games on it’s Wii U eshop via the Nintendo Web Framework

There are  clearly some emerging opportunities and avenues for game developers to pursue, all that’s left for us is to get out there and help grow some game-based initiatives in this soon to be fertile land!

...On Test:

For anyone still here, now we've seen the glorious colour bar charts and tables arrive, here’s our Graham with some juicy tidbits on the shiny machines we amassed to test on, (nb. we have a healthy spread of Nintendo, Sony and Microsoft fans in the office):

Consoles:
Xbox One:“The beefcake from redmond”
Internet Explorer 10, 8-Core AMD custom 1.75 GHz CPU, AMD853MHz GPU, 8GB RAM (480 MB RAM accessible to browser),  Includes in browser Gamepad support for up to 8 players!

Xbox 360: “You don’t have to put on three red lights”
Based on Internet Explorer 9, Xenon triple-core 3.2 GHz CPU, ATi Xenos at 500 MHz, 512 MB RAM

PS4: “The fashionista from Japan” 
Webkit based browser,  8-Core AMD x86 Jaguar 1.6GHZ CPU, AMD Radeon based 800MHz GPU, 8GB RAM

PS3:  “The Cougar who Could” 
Netfront Browser, Cell Broadband Engine 3.2 GHz CPU, NVIDIA/SCEI RSX 550 MHz GPU, 512MB combined RAM

Wii U: “Utterly devoted to you… Awkward identity issues at parties” 
NetFront NX Browser, IBM Power®-based multi core 1.23Ghz CPU, AMD Radeon-based GPU, 2GB RAM

Portables:
PSVita: “Touch me..if only to see if I’m still alive”
NetFront NX Browser, ARM Cortex A9 4-Core CPU, SGX543MP4+ GPU, 512MB RAM

New 3DS: “Personality with stable depth” 
NetFront NX Browser, ARM11 MPCore 4x @ 268MHz  CPU, 4x VFPv2 Co-Processor, 256MB FCRAM, 10MB VRAM, PICA200 GPU @ 268MHz

Mobile:
Nexus 5:  “Romantic Android”
Chrome Browser 40, Android 5.0.1,Qualcomm Snapdragon™ 800 2.26 GHz CPU, Adreno 330 450MHz GPU,2GB RAM

Apple iPhone 5S: “Fruity and flashy”
ios8 Safari, Dual-core 1.3 GHz Cyclone CPU, PowerVR G6430 Quad core GPU,1GB RAM

Desktop:
Dell XPS 15 9530 Laptop: “Dell-ighted to be here” (groan)
Google Chrome and Internet Explorer 11, Intel Core i7 2.2Ghz CPU, Nvidia GeForce GT 750M GPU, 16GB RAM

Macbook pro:“Professional lover... in a rain mac” 
Safari Browser 8.0.2, Intel Core i7 2.3GHz CPU, Nvidia Geforce GT 750M GPU, 16GB RAM