Designing Total War: Warhammer II to handle tons of units and massive battles

For venerable U.K.-based studio Creative Assembly, Total War: Warhammer II represents a massively ambitious undertaking.

Not only is it one of the deepest strategy games currently available, the recent (free) Mortal Worlds update integrates almost all of the content from the first game into the newest iteration -- including the stitching together of two massive, sprawling campaign maps, and bringing all of the diverse factions, units, and leaders from the original forward into the sequel.

The team also recently launched the first major DLC expansion, Rise of the Tomb Kings, which incorporates yet another race and another handful of factions to bolt onto the staggering 117 already present in the game. 

Managing all this content and making it behave together is a huge undertaking and one, according to Creative Assembly’s Al Bickham, and Scott Pitkethly, that’s taught their team a number of important lessons along the way.

“So much extra effort went into the development of Warhammer II,” Bickham says, “with attendant changes in so many areas. The databases and codebases for both games had diverged to the point where merging content from one branch to the other caused us a great deal of errors.”

Teaching an old dog new tricks

This was a particularly nettlesome issue when the team was integrating the Norsca Race Pack from the first game, which added a host of marauding northern tribes. The divergent codebases “pushed back our plans to integrate Norsca into Warhammer II and Mortal Empires, as we now have to re-implement all those individual content, data and code components by hand over time," adds Bickham. "An annoying – if useful – lesson learned!”

It’s a very specific example of the sort of issues that trouble such a huge project, but some of the initial hurdles were much more banal and commonplace.

One of the big challenges of a game of Warhammer II’s scale is rendering thousands and thousands of units on screen at once, preferably with as little a performance hit as possible. According to Pitkethly, in designing the game they tackled the problem with a multipronged solution.

“[Dynamic] environment and unit LODs [Levels Of Detail] obviously help a great deal from a GPU-load perspective, and we have some clever LOD techniques to reduce our resource requirements," he said. But level of detail solutions only partially resolved the issue, and in a game that enables dynamic zooming and camera rotation, additional techniques were mandatory. “Total War: Warhammer also saw us switch to multicore processing in earnest, so we distribute tasks across a greater number of CPU threads, and run them in parallel.”

The real secret weapon in Creative Assembly’s arsenal, however, seems to be a brilliant sort of prognostication.

“In Total War, the battle logic and animation pipelines (both of which are very CPU intensive) have also been decoupled from the display, allowing them to run simultaneously,” Pikethly says. “In a battle scene, the logic generates the 'future' whilst the display renders the 'now'. This future logic-state can be created over many display frames, allowing us to calculate complex interactions without impacting frame rate.”

Another difficult quandary presented itself as the team was designing and integrating units of different types. When your game includes flying units, infantry, cavalry, artillery, commanders, and fantastical beasts ranging from animate stone giants to ravenous horned demons, it becomes a delicate, complex balancing act ensuring they all interact appropriately with one another on the battlefield (and sell the damage done to them by other units in believable ways). Unsurprisingly, it’s another problem that requires a suite of solutions.

“There’s quite a number of techniques we employ, often with bespoke approaches to cater for different situations," says Pikethly. "Some solutions are based on animation-set choices for example; we may author – or choose to cull – specific animation-sets for units fighting atop, say, a castle wall, in order to avoid weird-looking situations or extreme movements which would otherwise be fine on open ground. We may then capitalize on certain situations by authoring matched fighting animations between specific entities.”

For Bickham, sprinkling in these little moments is a crucial way to make players feel as though they’re witnessing a real, dynamic fight, as opposed to a lot of complex math.

“A good example in Warhammer II is when a Carnasaur [basically a giant T-Rex!] fights a grounded dragon," he says. "In general they’ll trigger their usual attack animations, but in rare situations you’ll see the dragon clutch the Carnasaur and try in vain to fly off with the thing, before being dragged back down to the ground and counterattacked with a hefty head-butt. These occurrences are rare enough to make them feel special and epic when they do happen.”          

The issues surrounding unit interaction are further compounded, however, when uneven terrain or structures are involved, particularly in the case of some of the game’s enormous, signature sieges. There’s a colossal amount of data to process in these situations, particularly, Pikethly says, because of the way Warhammer II models combat.

Matching up what the engine sees with what the player sees

“We physically simulate every projectile fired by each individual soldier or entity, and accuracy is variable according to the firer’s quality and the target’s situation. And every building and terrain-feature provides a potential obstacle. Calculating those shots en-masse is expensive, so we have to be clever about how we request such checks in order to keep performance high.”

And it’s not just a matter of properly optimizing, managing, and queueing combat result checks, there’s also the issue of line of sight. Pikethly asserts that one of the biggest problems for their team is the difference between what a player sees and what the engine is actually rendering.

“From a sky-eye view with your whole army visible, a tiny wrinkle in the landscape between your riflemen and their quarry may be barely discernible; to the player, the riflemen appear to have line of sight. From a soldier’s-eye view however, that wrinkle may actually be a low hill the height of a bus-stop, blocking their line of sight. So when the player orders them to fire, they begin moving forward into a position where they can draw a bead on the target, leaving the player baffled as to why they’re marching blithely towards the enemy instead of firing as ordered!”

And that marching bit presents its own difficulties in the form of AI pathfinding, specifically when a unit has to pick its way through broken terrain or a complicated urban environment.

“Our Battle AI relies on hint-lines (invisible to the player) around hills, for example, to tell it that the hill is a desirable feature to hold, and where best to position its forces,” Pikethly adds. “Likewise, a poorly-connected network of city-streets might be impossible for the AI to negotiate. The player may see an open junction between two roads, but the AI sees an impassable area, and when ordered to move, may take what seems to the player to be an inefficient route.”

The solution to all of these problems comes largely down to design. The advantage of hand-crafting maps rather than relying on procedural generation is that, while more production work is involved, every piece of terrain can be sculpted and placed and assessed according to how it fits with every other piece of terrain, structure, or pathway. Linking streets in an orderly, sensible way is key, as is placing impassable terrain so that it won’t halt an AI unit marching around it, or avoiding terrain features that present significant LoS challenges but aren’t visible from a bird’s-eye perspective.

“Battle-maps and the terrain features on them require diligence in execution,” Bickham says. “We code the AI to understand how to negotiate terrain and buildings and are very mindful with map design from both a gameplay and a technical perspective.”

Throughout the process of integrating old with new, it's clear that Creative Assembly can chalk up a good bit of its success to careful planning -- as well as thoughtful analysis of what's most key in giving players the intended experience (in this case, simulating believable, massive skirmishes) and making design choices that support those priorities.