AI and Games is a crowdfunded YouTube series hosted on Patreon.  If you enjoy reading this piece please consider supporting the show at www.patreon.com/ai_and_games

In 2016 Tom Clancy's The Division brought players together in the war-torn streets of Manhattan fighting against a variety of enemy factions, as well as one another. Ubisoft's RPG-shooter carries a diverse range of enemies to keep the player on their toes. But unlike other games I've covered to-date, The Division needs to deal with the practical issues of being an online co-operative game and how to manage all sorts of AI systems for thousands of players across the world.

In this first of a two-part series, I'm going to be looking at how developers Massive Entertainment sought to create a variety of interesting and challenging encounters throughout the world of The Division; how they designed their enemy opponents and manage their behaviours as thousands of players rush to battle online every day in the mean streets of New York City.

The Division

After an outbreak of weaponised smallpox, New York city breaks down into chaos and is quarantined from the outside world until the contamination is contained and order is restored. Players assume the role of an agent within the Strategic Homeland Division: a sleeper government unit that is activated in periods of national emergency in order to help first responders bring order to the region. Players can play individually or in groups of up to four completing a series of story missions and open world events in a region of Manhattan that at launch spanned over six square kilometres in size ranging from 21st street and 9th all the way to 55th and 3rd avenue.

To give players some variety and challenge, regions of the map are controlled by five rival factions: including the flamethrower-loving Cleaners, escaped convicts from Rikers Island, as well as the private military company the Last Man Battalion. In addition the most infected region of the city - The Dark Zone - has been cordoned off and is the games major PvP arena comprised of not only enemy AI but also fellow players who can go rogue at any time and begin to attack you.

As players explore the city of New York, they will interact with non-player characters (or NPCs) in one of three scenarios:

In each instance, the AI placed in the world has specific objectives that are aligned with one of two critical design goals:

The latter was the critical part of the game: given it aims to encourage players towards scouting areas prior to combat and establishing a base level of strategy. Once fighting breaks out, players need to continually re-evaluate their combat priorities and spatial awareness in order to come out intact. This requires enemy NPCs to carry interesting behavioural traits for players to recognise, but also some consistency in their basic behaviours, with a variety of subsystems that handle threat detection, dynamic objective handling as well as being able to react to players themselves.

The System

All NPC behaviours are achieved through use of a behaviour tree system, where subsections of the tree are activated to run specific behaviours that are seeded with certain objectives or variables. This can include the likes of target destinations for navigation, areas to explore potential threats, enemies to attack, cover to move into and so-on. I'm going to look at the design and structure of these for enemy NPCs, but if you want to learn more about how behaviour trees work: check out my articles on the AI of Halo 2 and 3 which cover some of this in more detail.

The behaviour tree architecture shown above (note: read from right to left) was reliant on a richly layered system, which was capable of providing multiple paths to each behaviour. This is because of the range of ways in which one specific action could arise and required the system to be as flexible as possible. The top layer of the tree [yellow-right] breaks down the three critical ways in which a NPC would react, including direct orders from in-game systems, to maintaining active combat or initiating combat if necessary. This moves down into the conditions layer [blue/purple-centre], which identifies what behaviours can be executed based on the current conditions of the world and the character itself, all the way down to the lower layer [left] which handles pretty much every variation of combat that is possible in the game.

What's critical here is that the middle layer is built such that it can link to a variety of different actions on the lower layer. This is to prevent behaviours being duplicated through the bottom half of the tree based on very specific conditions. As such all the different NPCs in the game (which I'll come back to in a minute), as well as tools such as the Seeker Mine - which is technically an AI character - can have similar sub-trees or behaviours to suit their purposes which are copied around. As you can imagine, this would result in a behaviour tree architecture that is rather complex - and you're right - what you see below is simply massive in scale and this required heavy collaboration between programming and design teams in order to get this working. Given the scale of the problem and the challenges faced, the programming team developed numerous tools for editing and eventually debugging each of the NPCs and built these into the Snowdrop engine that The Division is built upon. These debugging tools are extensive, given they allow for the developers to identify what is going wrong with a particular character in real-time in an online instance and I'll come back to talking about this later.

So now that we know a little more about the AI architecture, let's take a look at the design of the NPCs themselves.

Archetypes & Factions

At launch The Division had 36 enemy NPCs types spread across five factions. Archetypes help establish the type of behaviour of a given character, while factions govern attributes that are specific to all NPCs within them. To enable a bit of gameplay diversity, there are large number of archetypes in The Division:

There are also Special NPCs that do not adhere to the archetypes model - but these are only used in end-game and boss battles.  Each of these archetypes are designed to give clear feedback of the immediate dangers to the player and how to strategise their attack patterns. In addition, many of these archetypes compliment one another (as shown in the image below) such that one can exploit each others attributes in order to achieving their goals.

For example, while snipers maintain their distance from players to deal large amounts of damage, rushers deal damage up close and immediately try to close the distance between themselves and the player. Meanwhile Tanks and Heavies force players to stay in cover, while throwers will attempt to flush players out. As such this forces players to use different tactics throughout a given battle based on the enemy unit composition. This is driven by a desire to force players to continually evaluate three key gameplay attributes:

Factions

As mentioned earlier, the 36 NPC types were spread across the major NPC factions: Rioters, Cleaners, Rikers and the Last Man Battalion that is split into tiers 1 and 2. Depending on where you are within the New York map, you will encounter one or more of these factions in combat. Each faction is designed to scale difficulty as you progress through the game, progressing in the order I listed them. But given there so many individual NPC types, how do you manage the difficulty progression in a manner that is easy for designers to iterate upon, but also effective enough to be pervasive across entire groups of NPCs.

To achieve this, Massive built an AI profile system that modifies a number of exposed attributes about each of the NPCs. This enables them to become more or less of a threat to the player accordingly. This are eight different attributes that are influenced here:

The profiles that are built can be consistent across either a specific NPC type or a faction that NPCs exists within. So for example the Rioters - who are the first faction in the game players will encounter - are restricted to using a 'Dumb' profile where all of these attributes are fairly low. However, all Heavy's, Support's and Turrets have dedicated profiles for those archetypes that are consistent across all factions. Meanwhile the Last Man Battalion have their own Elite profile: which makes them far more threatening to players than all others.

The last and final modifier that has an influence on an AI's behaviour is it's role and rank within the game itself. Characters range from regular grunts to veterans, elites and finally the named NPC rank, which is reserved for only the most high-priority targets. As a NPCs role increases, this has an influence on base stats such as weapon damage, armour and shot accuracy, with elite and named NPCs having access to unique weapons and mods. They even get custom laser sights that have absolutely zero effect on their gameplay ability, but it puts players on edge as a result.

Closing

This suite of management systems enables Tom Clancy's The Division to build up a rigorous and diverse collection of non-player characters that help bring Manhattan to life. Whilst these systems fundamentally use the same core AI architecture, the use of archetypes, factions, profiles and ranks enable a lot of diversity in not just the behaviours executed, but also the level of challenge that these characters present for thousands of players across the world. But therein lies the bigger problem: how do you maintain performance for these kinds of systems in a game designed to run online for players all over the world? The Division has hosted over 10 million players since launch with a peak of over 1 million concurrent players. That's a lot of non-player characters that need to created, maintained and controlled over thousands of individual game instances.

In part 2 of my case study of The Division, I'm going to look at how Massive Entertainment pulled this off and explore the online integration. The design decisions that were made to ensure the game runs as smoothly as possible and the problems and limitations this presented for the AI systems. In addition, we'll also look at the tools that were built to enable testing of the AI behaviours in online instances, as well as how AI players were used to test the consistency and stability of story missions on test servers.

Bibliography