Welcome to part 4 of my series on the AI of Total War. In part 3 I explored the campaign AI of one of the most pivotal entries in the franchise: 2013's Total War: Rome II. A game that completely re-built the campaign AI systems to accommodate for an increasingly more complex series of mechanics, resources and consequences. Rome II's adoption of the Monte Carlo Tree Search (MCTS) algorithm is a critical step in bring the campaign AI up to spec for more contemporary entries in the series. But the innovations at campaign level didn't stop there. In this entry I'm going to look at how the MCTS systems were improved upon, as well as how the diplomacy systems have been scaled up for the modern era as Rome gave way to 2015's Total War: Attila.

Total War: Attila

Attila is the ninth entry in the Total War franchise and transposes the conflict to the late 4th and early 5th century: an phase of history known as the Migration Period. The Roman Empire begins to falter, even fracture; the empire is separated with the Latin-speaking Roman Empire to the west and the Greek-speaking Byzantine Empire to the east. The Roman Empire is on borrowed time and will only survive another 200 years at best as it is besieged by the vandals, franks, Saxons and perhaps most importantly, the Huns. Players bring Attila the Huns rise to power as the Hunnic empire forms over what is now eastern Europe. Total War: Attila is in many respects a natural successor to Total War: Rome II - with a campaign map of similar size and shape that succeeds the rise of the Roman empire in the previous title, only to see it crack and crumble with a total of 56 factions appearing as either ally or foe in the original version of the game.

Improving the Campaign AI

But of course, it's not just the change of setting, the underlying tech continues to change and evolve as well. The introduction of Monte Carlo Tree Search in Rome II, the focus of part 3 of this series, was a massive undertaking. MCTS had not made the transition into AAA gaming at that time and naturally there was still much to learn as to how to utilise the systems full potential. So let's take a look at the gradual changes and improvements made to the algorithm between post-launch updates for Rome II, all the way the launch of Attila.

The MCTS was updated to address some notable issues in performance and results. Notably, the algorithm was slow to decide, but was typically making good choices. A big part of this was that state space coverage becomes increasingly difficulty the longer the game progresses: the number of AI factions begins to grow, the number of possible future states grows at an exponential rate. In addition, the underlying pathfinding tools to calculate distances between factions and the like during MCTS playouts was proving very expensive. To resolve this, the algorithm was trimmed to prune the number of strategies it considers. This required additional analysis of a given state using metrics from the developers itself, to determine whether a target was unreachable, a battle was unwinnable, there was a low probability of a strategy succeeding or a given path in the tree was redundant and then removed it from the decision making process. This last part isn't as easy as it might sound, given a lot of strategies are essentially identical: executed in a slightly different order and result in a similar outcome. This required the system to break up searching into sub-phases such that is enforced an arbitrary ordering (see image to right from [Andruszkiewicz, 2015]). This helped identify multiple strategies that essentially the same when ordered a specific way (see below image from [Andruszkiewicz, 2015]). In addition, this all required aggressive patching of the pathfinding system to pre-cache some calculations and simply reduce the number of calculations taken, only asking if the MCTS decision making process deemed it necessary.

This resulted in a system that was more memory efficient and ultimately faster. It shows that the MCTS algortihm, while relatively simple in its design and execution, is also once that requires additional care when tackling large and complex problem spaces. Given it may need some assistance in order to make its decisions more efficiently. Especially when trying to ship it in a AAA product.

The Diplomacy of Total War

One area of the franchise I have not yet touched on is the notion of diplomacy: the process by which campaign AI players accept, offer and negotiate trade deals and alliances. Diplomacy is driven by an entire AI system in and of itself. Adding to the steadily growing collection of AI subsystems such threat analysis, path-finding and siege battles, which I will cover in part 5. Diplomacy is primarily interested in answering three key questions for the campaign AI on a given turn:

These questions are valid both for an incoming or outgoing deal, given it wants the other faction to accept its proposal, but conversely it may need to haggle an incoming proposal to its satisfaction.  The Diplomacy system is driven by the same information and logic as the player, it can't take any shortcuts, even when two AI factions are dealing with one another. They still go through the same process as a player would either with an AI or another player. This is a highly data-driven process, with each faction having specific and unique configurations of values that drive diplomacy in an effort to ensure they all behave slightly differently. This is pretty important in deriving their personality throughout a given campaign, which I'll come back to discuss a little bit later.

Deal Systems

In order for the AI to pull it off, it transforms these three key questions into diplomacy sub-systems: deal evaluation, generation and negotiation. Each of these systems utilises four key metrics that are generated by the system given the incoming information it has received:

If diplomatic value is negative, it means that everyone else really won't like you signing this deal. In addition, things like stance and strategic value are influenced by a balance factor that is hand-tweaked by designers. The balance factor is incorporated into the difficulty and in-game progression, such that when an AI is losing badly, it'll be more likely to accept offers that might not even by that useful, even from people they don't really like. Given it may help ensure their survival.

For deal evaluation, all of these values are then pumped into a weight summation function to give what is called the Deal Value: if the deal is scored as higher than 0, it means that it is worth the AI player accepting that deal.  Meanwhile in deal generation, it needs to make some smart decisions on what to offer. The list of all possible diplomatic actions is took long to consider, as such the system will use a pre-filtered list based on the current strategic situation, factoring strategic and economic values pertinent to the current state of the game. It then prioritises each deal by evaluating it, with the actual evaluation shifting throughout gameplay, given that a deal may have more or less value at different times during a given game.

Lastly, the system negotiates deals by starting with a collection of generated deals that it evaluated as useful. It will then begin to offer these to other players, but is mindful of previous offers it has made with a given faction. If it receives an offer that the deal evaluation scoring disliked, then it will make a counter offer with a given probability or simply reject it. The AI players are made to weight a certain number of turns before attempting diplomacy with a given faction again, so as to avoid spamming you with new offers every 2 minutes. The actual discussion between faction leaders that takes place during negotiation is powered by a separate system that ensures that the dialogue fits the style of the people of a given faction.

AI Personalities

Speaking of AI factions, by the time all of the DLC was released for Attila, there was over 80 factions in the game. So how do you make all of this diplomacy not feel stilted or samey with each and every faction you deal with? As mentioned before, the diplomacy system is data-driven, so each faction is fed data that influences how it behaves in areas such as budgeting, diplomacy evaluations and negotiations and technologies. These components craft what Creative Assembly consider to be AI personalities. Each of these components are often quite extreme and binary, with budgeting making the AI range from Scrooge McDuck to Kanye West. Each of these personality traits are effectively communicated via the user interface, such that players can establish just how to deal with a given faction.

But it's not enough to have just a static personality throughout a given campaign, it needs to change and evolve over time. As such, each faction has a personality group with multiple personalities it can choose from. The choice of selection which personality it wants to use is driven by a number of factors, such as the current difficulty of the game and the stage of the campaign itself. As such, ultra-aggressive personalities don't appear as frequently at the start of the campaign, but are more likely to appear towards end-game. In addition, many of the personality swaps are tied into the progression of faction leadership: with new personalities adopted as a faction leader dies and their heir comes to power.

Closing

While deciding how to conduct war can be challenging, so to can be forging alliances and making peace with others. The scope and complexity of Total War continues to be vast in scale, to a point that the number of minor AI systems continues grow in order to support the unit, battle and campaign AI. But diplomacy isn't enough in times of conflict, sometimes we just need to force our point of view on our enemies. To make them bow to our will. In part 5 of my exploration of the AI of Total War, I'm going to look at another of the AI subsystems in play: one that is responsible for laying siege to an enemy fortification. While these have been critical to the series for many iterations, I'm going to take a look at the more recent innovations brought to this system alongside some fresh perspective for the franchise: as history is cast aside for fantasy and we enter the world of Warhammer.

Bibliography