Since the beginning of my AI and Games series on YouTube I've explored a variety of AI techniques to solve specific problems within video game design that require both immediate, reactive behaviour in conjunction with long-term and strategic decision making. First-Person Shooters (FPS) are certainly a common genre from which interesting AI can emerge given its predicated on fast, reactive and engaging gameplay. But if we're to consider the need for effective systems that balance both short and long term decision making in games, there is no greater problem space to explore than Real-Time Strategy (RTS) games.

Both the RTS and FPS genres have continued to innovate the state of the art in AAA video gaming in an effort to satisfy their design. But RTS games have much larger considerations for the long-term impact of quick and reactive decisions, as well as the need to manage resources at varying levels of abstraction and complexity over multi-hour gameplay sessions. When considering the challenges and innovations faced in AI for strategy games, as a player, developer, or researcher there's one franchise you simply cannot go without discussing, Creative Assembly's Total War.

Total War has grown from strength to strength since the launch of the franchise in the year 2000. As each game becomes more ambitious, the challenges faced by the AI systems continue to increase in scope and complexity. This has resulted in a development team that continues to push the state of the art in game AI unlike any other in video game history. But the journey to reach more recent entries at the time of this video, such as Total War: Attila and Warhammer has had more than a few bumps in the road along the way. So in this first in a series, I will be looking at not just how the core tenets of the AI of Total War, but how each entry in the series aims to solidify, tweak or completely rewrite parts of these systems. In addition, I'll look at the response to these changes from the fanbase, the modding community that has thrived within Total War and how The Creative Assembly adopted groundbreaking academic research to help innovate from the very first entry all the way to more recent releases.  .

RTS Games and AI

But first let's look at the Real-Time Strategy genre and understand where Total War sits within this space. Real-Time Strategy games are focused primarily on the control of territory and assets: whereby two or more opposing forces will appear in the same map in an effort to assert dominance by securing resources or locations within the existing defined area. Often in order for this to be achieved, each force will build collections of structures and units that either improve the resource collection within a given area, defend existing structures and land or scout and secure areas of the map by force. The latter often requiring the destruction of opposing forces units in the process. This process typically requires players to explore a previously hidden environment by piercing what is referred to as the 'Fog of War' in order to establish enemy and resource locations. In addition, over time each faction is expected to build upon the structures and units previously established by enhancing their existing capabilities and resource acquisition.

While arguably not the first RTS game ever made, Westwood's 1992 release Dune II: The Building of a Dynasty acted as the archetype for future games in this genre such as Westwood's own Command and Conquer series, but also Age of Empires, Total Annihilation, Dungeon Keeper, Homeworld and Blizzard's Warcraft and Starcraft franchises. The genre has continued to evolve and branch off since the late 1990's, with the likes of real-time tactics games like Dawn of War and Star Wars: Empire at War emphasising tactics over resource management. Meanwhile games such as 1994's UFO: Enemy Unknown have explored a balance between the real-time and turn-based decision making, with it still proving popular to this day with the franchise being successfully rebooted as XCOM: Enemy Unknown in 2012. Arguably the largest deviation is the the rise of the Multiplayer Online Battle Arena or MOBA genre - popularised by the Warcraft III mod Defence of the Ancients and typified by games such as Dota 2 and League of Legends - where players assume the role of hero units on the ground to define and assist the larger strategies playing out during a match.

AI applications within real-time strategy games have existed since the inception of the genre, with players responsible for managing the movement of faction units by telling them where to go, but without direct control of their actual movement. As such, players are expected to interact with these systems in order to yield desired outcomes. In addition, AI is critical to the single-player aspects of the game, given that players play off against opposing commanders responsible for managing their own units and construction strategies. The role of the AI player is one that is tremendously challenging and is considered to be one of the larger challenges facing academic research. This has led to the forming of numerous academic benchmarks and projects, including the Starcraft AI Competition hosted at the AAAI conference on Artificial Intelligence and Interactive Digital Entertainment. This is a topic in and of itself, but don't worry: I'll be talking about that on AI and Games in the near future.

Total War

So what makes Total War so special and challenging for an AI system? Creative Assembly's strategy series is interesting in that it combines multiple modes of strategic gameplay into a single game: with players managing turn-based resource management and political strategy alongside real-time combat with large scale armies. In addition, the combat juggles between micro and macro levels: requiring not only AI control of individual units, but more abstract control of unit types and managing the combat formations and troop deployments on the battlefield at runtime.

The first entry in the franchise, Shogun: Total War, balances the combat simulations that strive for realism and authenticity, alongside the political strategy that is aimed to give context and stakes to each conflict. The original Total War is set in 1530, during the Sengoku Jidai period of feudal Japan: a time largely popularised in contemporary fiction by the works of Japanese film director Akira Kurosawa, with films such as Kagemusha, Seven Samurai and Ran proving influential on the design and development of the game - with clips of the latter being used as part of the games cinematics. Both the player and opposing AI assume the role of the 'Daimyo': local-lords who control provinces of Japan with a need to conduct both diplomatic strategy alongside military movements. When rival factions are drawn into conflict, players take control of the 'Taisho' (General) and move hundreds if not thousands of troops across the battlefield. Sengoku Jidai made for an ideal period of history for the game, given the politics and even the economics of the period was built around the logistics of fielding armies in defence of the Daimyo and his ambitions.

Total War deviates from many traditional real-time strategy games in that it removes mechanics such as Fog of War from combat gameplay as well as the need for resource management outside of unit counts from combat decision making. Given the nature of the game itself, the AI player is comprised of two distinct AI systems:

So to get to grips with the AI of Total War, I'm going to explore the AI systems from lowest to highest level: from individual troop control systems all the way strategic systems that seek to conquer all of feudal Japan.

Unit AI Behaviour

Total War combat is driven by units: groups of specific troop types that can be deployed in formation ranging from melee types to archers to cavalry. These combat units are not only expected to keep formation, but to move around and conduct combat as a unit. Movement is often a tricky task, especially when asked to navigate through or around a variety of terrains such as mountains and forests.

Neural networks act as a weighted summation function of incoming data. By customising the weights between each simple compute node or 'neuron', we can change the result of the data being pass to the layers on the far right. [IMAGE SOURCE]

To achieve this, Total War adopts artificial neural networks for managing individual units. Neural nets are a fast and effective means to achieve quick and reactive responses to a pre-defined objective. A neural network processes data via 'neurons': simple processing units that take incoming data via 'synapses' that are weighted.  By changing the weights, we change the outcome of the data being passed through the neural net.  We typically train these weights using machine learning methods but they can also be tweaked by hand if the network is small enough. What makes this so effective is that once they are trained, the processing time to make decisions is lightning fast. In addition, a well-trained neural net is able to generalise its decision making process: meaning it can recognise the similarities in numerous individual circumstances and in each case give a similar answer.

Despite this, neural networks are typically not great when being given multiple objectives to complete at once, especially when they oppose one another. In this instance, the troops have different neural nets they can trigger in depending on the objective they're trying to solve, whether it's moving formation, avoiding oncoming enemy fire or taking position to attack themselves, be it up close or from afar. Each of these networks is pre-defined and is not tweaked or optimised during gameplay, so it's not learning to be better at the game as you're playing it.  These Unit AI systems are in effect regardless of whether a human or AI is in control of the army, as each requires these neural net systems to manage the individual troop behaviours.

Combat AI Behaviour

The combat level is the AI system that controls opposing forces akin to players. This AI needs to decide the group formations and actions of units. Moving segments of its army together in order to achieve a specific tactic or micromanage movement of units given the terrain or weather that is currently in play.

To achieve this, Shogun employs a logic system that allows for specific decisions to be made based upon the current state of the world. These decisions are derived from 'The Art of War': a collection of strategies by Chinese military strategist Sun Tzu that dates back to around 500 BC. The use of a logic system is well suited to adapting the texts, given that in many instances a set of conditions are in play that should result in a particular response from the system. These rules considers the type, condition and size of both our units and our opponents, the current terrain, the weather and state of troop morale.  Some simple examples derived from The Art of War are listed below:

It's been stated that there are 220 rules embedded in the combat AI system across three distinct versions, each with their slightly different sets of behaviours.  A driving force behind this was to ensure that combat was as realistic as possible, deviating away from the tactics that had driven many real time strategy games of the era. As Mike Simpson, Development Director of Creative Assembly stated:

Campaign AI Behaviour

The final layer of Shogun's AI systems is the campaign and diplomacy system. This is in control of all decision making on the main campaign map: managing army movement, conducting diplomancy between factions and building infrastructure within provinces.

This uses a mixture of classic state-machine decision making alongside the use of genetic algorithms in order to customise the tactics made by each Daimyo. The simple state-act system simply dictates that in a given circumstance, the AI will have an immediate response.  It doesn't carry any memory of existing behaviour or have any complex decision process.  It simply reacts to the events transpiring around it.  The genetic algorithm then allows for some customisation.

Genetic algorithms are a form of machine learning where we create large quantities of candidate solutions to a given problem that learn to solve it by breeding the best ones with each other.  These candidates aren't code, they're an encoding of how the AI behaves typically as a string of numbers, which is then translated into the AI itself to dictate how it behaves. These allows for us to create a collection of diverse solutions for a given problem: they don't all behave in the exact same way and have their own slight characteristics. As a result, each Daimyo has a unique combat behaviour. From a players perspective, this gives them each some sort of personality: allowing for Daimyo's that are more willing to conduct diplomacy to others that will take a more aggressive stance in expanding their territory: attacking both AI and human player alike.

Medieval: Total War

With these three systems, the core tenets of the AI of Total War are in place: to manage individual units whilst retaining cohesive behaviour, to group units of troops together in a manner that is responsive, reactive and challenging towards players and to create strategic opponents with personality. Despite being built back in the year 2000, the core principles of combat and strategy, combined with the systems built in Shogun have carried the Total War franchise forward into new and varied theatres of war in many a subsequent release. Despite these innovations behind the scenes - even from the very beginning - the AI has never proven infallible and as such, with every release there is a drive to tighten it up and improve it as best as possible.

The first changes came as the series migrated from feudal Japan in 2002, with the launch of Medieval: Total War. Medieval pulls the game farther back in time to the late 8th century all the way to the 14th century, with players taking charge of a variety of factions at different periods of history: with the likes of the Viking era in Britain - where much of the country is fractured following its abandonment by the Roman Empire - all the way to the late-medieval period, where much of Europe has taken shape as actual countries.

Medieval: Total War is in many respects a refinement of the original game and is built on an improved version of the original Shogun engine that among other things allowed for larger battles than its predecessor. There are no major innovations between Shogun and Medieval for the underlying AI tech, just a number of optimisations and improvements. The unit AI systems were improved to prevent bottlenecks in tight spaces such as bridges. In addition, the suite of combat AI systems was expanded from three up to nine allowing for greater versatility. All in all this resulted in a game that Rock Paper Shotgun referred to as "near-perfection". Rob Zacny says...

Closing

Shogun and Medieval were just the beginning of the franchise and it's safe to say that the road to war was not a smooth one. With each subsequent release in the franchise, the challenges grew in size and the scope of the games themselves increased significantly.  I'll return in part 2 with an exploration of the modding community that arose from 2004's Rome: Total War, how the release of Medieval II: Total War drove a need for the core systems to be re-written and rebuilt in Empire: Total War and how this seismic change impacted the franchise all the way to Total War: Shogun 2.