Optimising PixelJunk Shooter and giving it the “Ultimate” look

This month sees the release of the PC version of PixelJunk Shooter Ultimate from Double Eleven, which is a graphically overhauled and unified version of PixelJunk Shooter and PixelJunk Shooter 2 created by Q Games.  When we were trying to decide what a new version of Shooter should look like, we knew that it needed to stay true to that classic ultra sharp vector-based 2D style, which continually threw fresh visual ideas to the player while still being able to stand beside today's titles. As well, from a gameplay perspective we wanted to make combat more intense, engaging and overall provide for a more rewarding experience.

In this article I will talk in detail about some of the rendering enhancements we made, and in addition some of the major changes we made in the name of optimisation, so that PixelJunk Shooter Ultimate would be accessible to the largest possible range of PC configurations.

GPU optimisation - draw fewer pixels, draw cheaper pixels

PixelJunk Shooter and Shooter 2 are fairly GPU intensive games, for several reasons. The fluid simulation itself is largely calculated on the GPU, which takes up processing time before we even get to drawing anything, the game’s collision detection is also based on data generated by the GPU,  and rendering of the fluid body itself uses some complex pixel shaders, in addition to the rendering of the levels and the objects within it. As a result, and especially with the console ports in mind, we spent some time optimising the system by which the fluid was rendered. As is often the case the optimisation process was the sum of many small parts, but I will explain one of the most radical changes for you here.

The fluid simulation works much like a real fluid, with individual particles interacting with each other, albeit with far fewer particles than in the real world, but with some nice tricks to make it look like more. You can find out a lot more about it here http://fumufumu.q-games.com/gdc2010/shooterGDC.pdf. At the end of the day, the game ends up with an off-screen texture describing the fluids present at each pixel and a full screen shading pass was done to draw the fluid pixels onto the screen as you see them.

For the purpose of collision detection, the game already looped over all the fluid particles and partitioned them into a grid that covered the entire level, so that particles that are far away from each other do not need to be tested further for collision. We expanded on this, and created a grid which held a mask of different fluid types. We didn't need to know about individual particles, we just needed to know what types of fluid existed in each grid cell, so this took up very little memory, to be precise it was the logical OR of all the particle types present.

Next we took the original pixel shader, which was designed to handle all fluid types mixing simultaneously, and created variants of this shader for specific combinations of fluids; water only, lava only, water and lava, and all the other combinations. Many of these permutations could be greatly simplified to produce a much shorter shader.

Combined with the grid I mention previously, we were able to select the ideal shader for each portion of the screen, and where possible detect that there was no fluid at all. Of course it's possible for the player to mix up fluids wherever they can manage and our system would cope with that just fine, but we knew it would be impossible to get all fluids everywhere so we would definitely make a big saving.

The screenshot below illustrates the system more clearly. Here we have water flowing down into a pool of magma, cooling it to form rock. The blue, green and red tinted squares show where different fluid shaders are executed, and more importantly, that no fluid shader at all is running on the rest of the screen. In the blue tinted squares, a water-only optimised shader is used, in the green tinted squares a water and magma shader is used, and finally in the red tinted squares, a magma-only optimised shader is used. That's it, in a nutshell.

The different colour squares represent different shaders in action, chosen based on the types of fluids present in each cell.

The only slight fly in the ointment was that one of the tricks used by the fluid rendering system to smooth out the appearance of the fluid, is achieved by allowing each fluid particle to leave behind a 'trail', which fades away over a short period of time. This of course broke our grid optimisation because in the 'trail' we need to draw fluids even when there are no fluid particles at that location any more. So we needed to introduce some temporal coherency to the grid, in other words, once a certain type of fluid was registered in a cell, it would have to be 'remembered' for a while after. We didn't want to store a timer for each of the possible fluid types because that would have greatly increased the memory footprint of the grid system and made it slower to process every frame. In the end we came up with an implementation which only required a few extra bits of memory per grid cell. The gory details will be left to your imagination but feel free to reach out and ask if it is keeping you awake at night.

CPU optimisation - the best optimisation is doing less work

Of course, we also looked at CPU side optimisations for Shooter Ultimate, particularly in reference to our console ports. Although we spent a lot of time on low level implementational details we also looked for high level algorithmic improvements which will pass on a saving even to powerful desktop processors which might end up powering our PC version of Shooter Ultimate. 

One of the most interesting CPU side improvements took the grid idea discussed above and ran with it.  As I mentioned above, the original game partitioned all the fluid particles into a spatial grid, and then each grid cell can be considered separately and indeed, in parallel. Within the grid the collisions between particles must be resolved but also energy is transferred to give the effect in game (to give one example) of lava meeting water and creating steam as a result of the water heating up, and rock as a result of the lava cooling down. At first we just hand optimised the code which handled each grid cell as best we could.  Then it became apparent that if we understood what types of particle existed inside each grid cell then perhaps we could run simplified versions of the code. For instance, if the cell only contained one type of particle (water, rock, lava etc) then there was no need to lookup the thermal dynamic properties of each particle on a case by case basis. This sounds like a relatively small optimisation but given that each cell represents an n-squared problem (every particle vs every particle) there can be quite a large saving. Furthermore, there were bigger savings. For instance, if a whole grid cell was full of nothing but rock or ice, then since all those particles are fixed in position, there was no need to handle physical collisions at all. Going even further, when ice reached its natural temperature there was no need to propagate heat between particles, so we could effectively put the whole cell to sleep and not process it at all until something changes.   Although this kind of approach led to a bit of an explosion in code permutations, which can be a little tricky to handle, it gave good savings so was well worthwhile.  Making these kind of optimisations to an existing game is always a little nerve rattling when it can lead to subtle gameplay bugs so we relied on our QA department to help us detect any issues.

The 'Ultimate' look

One of the most striking differences between the original game and Shooter Ultimate comes from the introduction of real time lighting, and ambient occlusion style shading, along with the use of depth of field on background layers, overall creating a much deeper looking image.  When we set about doing this the immediate challenge was how to generate the information to do these effects from the original level data, which we didn’t want to have to fundamentally re-author. Luckily for us the levels were stored in a vectorised format which allowed us to scrutinise the shapes of the levels mathematically. Crucially, with the addition of some hand crafted heuristics, we could produce some sensible surface normals for the background layers. In the screenshot below you can see that the intestine walls now have defined 3D looking features which combined with point lights attached to the ship and other moving objects, produce a good 3D lighting effect.  The ambient occlusion type effect is generated in real time by applying an edge detection filter to the depth buffer.  As part of all of this, we also of course had to change the player’s ship from a simple 2D sprite into a 3D model, so that it could tilt and roll to really show off the new realtime lighting.

Before and after: the addition of real time lighting and depth of field creates a dramatic effect.

Can we make the fluid look any better?

The fluid simulation is one of the most memorable features of the Shooter games, and the fluid rendering was another area we touched upon and tried out various improvements. I would say that in the end this was one of the less successful aspects of the remake, this is mostly testament to the fact that what you saw in Shooter and Shooter  2 on the PS3 represented a very well tuned and polished system created by Q Games, which could not really be improved on a great deal without fundamentally redesigning the simulation itself. In the end we slightly improved the anti-aliasing along the fluid edges, and added real time normal mapping by extracting a normal field from one of the intermediate stages of the GPU simulation. In addition I tried various other things. I tried to add additional lightweight fluid particles, they were designed to be short lived particles which were generated when there was a sharp discontinuity in particle velocity, they would follow basic ballistic principles and expire after a short time. The idea was that we would get a more splashy feel when fluid is moving at high energy. Along similar lines I tried to introduce additional particles which were skinned to the real particles to give the impression of a higher fidelity system. All in all, none of these experiments made the cut because they didn’t add enough visually relative to the time it would take to setup and tune, and the cost to simulate and render.

Particle systems - artistic freedom vs runtime performance

Another aspect of our graphical overhaul included the creation of a new range of particle effects. Sometimes these replaced the original effects and other times they augmented them to give a greater sensory impact. Our new particle effects were authored in TimelineFX (http://www.rigzsoft.co.uk/) which is a great tool for authoring 2D or 2.5D effects. Although it allows the authoring of complex effects based on physics and parameter curves, the primary purpose is to bake these effects into an animated sprite for use at run time in the game. For Shooter Ultimate however I took a different approach, and wrote a realtime playback engine. This allows far greater flexibility because we can override parameters at run time to make the effects interactive (such as the emission angle, rate and speed), we can also rescale area emitters to cover designer specified areas, and we can produce particles in world space so for example, a booster trail can be left behind when the ship moves. This represented quite an investment of time but leaves us with a powerful bit of tech that we can look to use again in the future. In addition to creating runtime playback engine, I also created a “particle effect compiler”. What this does is generate C++ code which is specifically tailored to an individual particle effect. Because the authoring system is highly flexible, the generic playback engine has to support all sorts of options, and evaluate all sorts of curves, from linear to piecewise bezier and everything in between.  The particle compiler creates code which runs a particle effect in a way that is just as optimised as if it had been written by hand, cutting out all indirection and branching through options. Indeed, it is perhaps more optimised than what could be written by hand whilst being remotely maintainable; it eliminates virtual function calls because the concrete class type for each particle variable is known at compile time, simple variables (such as constant values or linear interpolators) are directly inlined, it can even leave variables uninitialised if it knows they are unused throughout. Needless to say all the different effects are simulated in parallel over as many CPU cores as are available. If we wanted to simulate these systems on a GPU as well as just render them there, we would just have to change the compiler rather than having to recode all the effects.

Autogenerated C++ code for the ship's shield effect.

You may wonder what the workflow looks like for these compiled particle effects. Each particle effect is annotated with a tag in the TimelineFX editor which tells the compiler whether we want that effect to have a compiled version generated. We want to reserve the compiled effects for performance critical effects only, otherwise we would be bloating our code base and increasing compile times. If there is no compiled version for a given effect, we fall back to the generic runtime engine.  The problem is, of course, that our artists and designers don’t normally compile code, indeed they may not have the tools necessary to do so on their machines. Therefore we do a date check to see whether the compiled code is newer than the TimelineFX asset (using a date and time compiled into the code). If the compiled version is older we fall back to the generic playback engine. So when an artist or designer tweaks a particle effect, they can test it in the game locally because the game falls back to the generic playback engine. When they are happy, they check in the asset, and our continuous integration system builds the code which generates the compiled version again, and next time they get a version from our build delivery system, it is back to the super fast compiled version of the effect.  Needless to say it took some time and effort to eliminate all bugs such that the compiled versions of effects looked exactly like the originals.

With this powerful new system in place our artists and designers were able to decorate the Shooter Ultimate levels with lots of powerful particle effects. For instance, look out for the light shafts, caustics, bat swarms and steam clouds all of which were powered by this system, in addition to explosions, ship shields and other spot fx.

Conclusion

One feature that is exclusive to the PC version of Shooter Ultimate is the ability to switch backwards and forwards between the original ‘classic’ art style and the new ‘ultimate’ style. This really gives everyone the opportunity to see all the changes we’ve made!

In summary I’ve talked in detail about a few of the things we did on this project, both to create the Shooter Ultimate look and also to improve the game’s performance on all platforms. I haven’t really spoken about all the micro-optimisation we did, the host of other graphical changes we made, the extensive re-engineering of the dark level lighting, the trials and tribulations of network multiplayer, the overhauled UI, and much more besides. Nevertheless I hope you have enjoyed reading a bit about it as much as we enjoyed making it. If you have any further questions feel free to fire them our way on Twitter.