AI and Games is a crowdfunded series about research and applications of artificial intelligence in video games.  If you like my work please consider supporting the show over on Patreon for early-access and behind-the-scenes updates.

Horizon Zero Dawn stands as one of the most critically acclaimed of Sony’s roster of Playstation 4 exclusives. As the hunter Aloy, players venture across the post-apocalyptic landscapes of the future to uncover the mysteries of her past and how the world fell in the years before. Humanities fall from grace has led to the rise of ‘the machines’ – robots of varying shapes and sizes that that now run free across the lands. These robotic animals are intelligent, coordinated and lethal: requiring quick thinking to survive and even greater preparation and planning to successfully destroy.

In this first of a two-part series: I’m exploring the gameplay and decision making systems that bring these creatures to life and power the core gameplay experience. I’ll be looking at not only how they make decisions to scavenge, to hunt and to survive but how they coordinate themselves in the wider world either individually or as part of a herds. Plus I explore how the game creates an ecosystem of machines that is balanced and managed across the many territories and regions of the new world.

Warning: Story spoilers for Horizon Zero Dawn.

The Ecology of Horizon Zero Dawn

The robotic wildlife of Horizon Zero Dawn dominate the landscape and hold control of territory throughout this open-world experience. Each machine carries a unique aesthetic and structure that reflects their function and purpose within the narrative of the game. As players work their way through the campaign, we come to learn of the Faro Plague – an army of rogue war-machines that rendered the world asunder – alongside the race of machines constructed by the AI known as GAIA that are designed to restore the planet to its natural glory and are largely inspired by long-extinct terran species.

As a result, the function of a given machine often holds the secrets to their weakness, players can use their Focus to scan local machines to ascertain not only their type, but vulnerabilities as well as weak points. In addition to the weak spots, a machine’s function within the narrative ultimately influences their in-game behaviour, this includes:

There are 28 unique machines that can be found roaming the wild lands, each with their own distinct passive and aggressive behaviours and players need to learn how best to approach them. While the greatest threats on the path to Meridian – such as the Stormbird and Thunderjaw – roam the wilds on their own, players can still run afoul of less lethal machines that operate in herds. It’s common to see the smaller robots such as Watchers, Striders, Scrappers and Lancehorns co-existing: supporting one another as they achieve their base functions. Over time these herds increase in size and diversity as players continue to explore the world with the likes of Shell-Walker convoys, packs of Tramplers and Sawtooths protecting nearby Grazers and Lancehorns.

In order to pull this off, the game requires numerous AI systems that dictate the behaviour of each robot, how they behave within a herd, the collective intelligence of the herds themselves but also balance the herds across the open world for a more varied experience.  The developers behind Horizon Zero Dawn, Guerilla Games, came to this project lacking many of the AI tools required in order to bring this experience to life.

Having previously worked on their Killzone franchise, their toolchain was largely focussed to accommodate first person shooter and none of the design challenges that faced them for a large open-world experience. As such, during the course of development they built newAI and gameplay systems and expanded existing ones within the studio to bring this experience to life and over the next two blog posts I’m going to explore the following:

So let’s begin by exploring how each AI character is built to make decisions and how the game manages herd behaviour.

Agent Hierarchy

Each machine is represented in the world as a separate non-player-character AI called an ‘agent’, but perhaps more interestingly, so is each herd. Horizon Zero Dawn adopts a hierarchy whereby a machine can exist entirely on its own as an agent or as a child of a group agent – which is used to store information about herds.

An example group hierarchy [from (Beij, 2017)]

A group agent doesn’t physically exist in the game world, but it’s responsible for coordinating all agents within it. Each individual agent has its own set of sensors that enables it to hear noises closeby or spot threats in proximity. But in addition, a group agent has a blackboard system that stores in-game information a machine might want to use. This not only allows the game to keep performance costs down by storing information each individual might try to calculate on its own – like safe areas to stand in or active patrol routes – but more importantly, agents in a group can share information between each other. Hence when a recon machine spots the player, each machine can make a decision on how to respond to that information. Hence you might see acquisition robots in the herd such as grazers opt to run away while nearby combat machines move in for the kill.

Now the actual AI system used in Horizon Zero Dawn is called a hierarchical task network planner. Unlike the planning systems used in games such as F.E.A.R. that planned one action after the other, HTN planning generates plans comprised of action macros. Each macro contains multiple actions in a set sequence. This is ideal for game development given designers can put together macros of good behaviour they expect to see in-game. I’ve previously explored how HTN planning was used in Transformers: Fall of Cybertron, but it’s worth noting that Guerilla have been used HTN planning since Killzone 2 and as such this wasn’t built overnight, but has gradually matured over years of projects within the studio.

An overview of HTN planning [from (Beij, 2017)]

The HTN planner is used in two capacities by machines: individual agents can ask for a plan of action to resolve a specific task, such as finding new areas to visit, attacking a human, or fleeing to safety. Meanwhile group agents can run plans that dictate what goals individuals agents should be assigned, allows groups to actively pass information between one another within a hierarchy and potentially form new subgroups or destroy existing ones. In short, each individual machine is still largely responsible for their own behaviour and addressing the goals assigned to them and uses the planner to make decisions. Meanwhile, the groups keep individuals coordinated, re-group them based upon what’s happening in the world and – perhaps critically – assign roles to each machine such that they know what their job is to support the herd.

The Collective

Now with all of these machines being spawned into the world either individually or in groups, how does the game know what machines to spawn in the open world and where? There are designated sites across the game map where we can expect specific machines to appear either individually for machines like Thunderjaws and Stormbirds or in flocks for the likes of Striders, Grazers and Glinthawks. However, the rest of the game-world is full of herds comprised of different machines. So how does the game know when spawning in a specific machine where to place it in the world and what groups to assign it to based on the needs of the worlds design?

This is achieved courtesy of a system known as ‘The Collective’. The collective is the supergroup: all machine groups and individual machines exist in the collective. It manages the spawning of all machines into the world, keeps track of whether they’re in a group and handles their movement between groups and manages the machine ecosystem, such that performance isn’t impacted overall.

Individual machines in the world can request to join an existing open-world group if they’re on their own. This is useful feature given many machines are left isolated after the rest of the previous group either fled or have been destroyed. So the collective can recycle isolated machines to join other groups. To do this, each machine has a passport! Their passport stores facts about that machine, such as its level and machine type and the collective uses this to determine whether it will fit the requirements of another group, only moving across if it satisfies the groups needs.

Herd Example

Now let’s explore how herds operate and how group agents execute as players interact with them. Herds are typically placed into the world in a relaxed state where they’re not under threat and simply hanging out in a specific region of the map. Any given herd is typically a mixture of recon, acquisition and combat machines, with acquisition robots in the centre, whilst combat and recon patrol the outside to defend them.

When a herd is placed into the world, a collection of groups within that hierarchy request a role. This role assigns all machines it’s controlling with behaviours that reflect their purpose. This can include patrolling for threats, searching nearby disturbances, attacking enemies or simply scavenging resources in the world. These roles can shift as time passes and events force the herd to change its behaviour and structure. There is typically a limit on how many machines can fill a particular role within a herd such that their make-up isn’t unbalanced.

A machines role dictates the goals that the an individual agent will seek to complete. Hence with a relaxed herd, the acquisition machines will use the HTN planner and data stored within the group to find a local area to mine. Meanwhile recon and combat machines will generate patrol paths to follow. Patrol paths are auto-generated based on the local geometry, avoiding awkward terrain and – more importantly – avoids passing through long grass and other vegetation the player might use to stealthily approach the herd. However, in order to give players a fighting chance, many patrol paths will deiliberately pass close to stealth vegetation such that you can place traps or attempt to hack them more easily.

Herds rebalance their composition when under threat, forming ‘flee’ and ‘combat’ groups [from (Beij, 2017)]

But of course it doesn’t always stay this way; as players start to attack machines their behaviour will shift accordingly. For machines that aren’t part of herds, this comes down to the specific machine class. Only a few will actually run away when provoke directly, with the vast majority of machines attacking the player, with even acquisition machines often attacking the player if they’re on their own.

However, should the player attack a herd, the groups within hierarchy react in kind, but only if the herd has been alerted. As mentioned earlier, the group agent enables data to be shared between each machine, but it doesn’t update on every tick. As a result, the herds don’t have a hive intelligence and it is possible for the player to kill a machine without alerting the rest of that group. However, should this be spotted or if the machine under attack survives, it will notify the group that they’re under attack.

At this point, the hierarchy within the herd shifts, groups of machines within the herd request a new role and priority for execution of the corresponding behaviour. The group comprised of acquisition machines assume the flee role and run away together, while the groups comprised of combat and recon machines re-balance themselves with their roles prioritising each character that is attacking the herd. While in many cases this is just the player on their own, this enables the system to scale for story missions where the player is part of a group or for other dynamic events in the world where other human NPCs are attacking herds.

While at this point things have gone horribly wrong, it still gives players plenty of chances to turn the situation around, as both the grouping systems and AI combat systems work to create an experience that is both fun and challenging. This is achieved courtesy of an AI combat system that attempts to structure combat such that it isn’t chaotic or broken.

The combat groups that formed have different roles and priorities that are balanced out, hence the larger machines in a herd will be spread across the groups where possible, with smaller machines lined up to support them. In addition, groups select machines to attack the player based not just on whether the HTN planner thinks an it can execute a specific behaviour, but also whether that attack is going to be interesting. If all machines simply attacked the player at once, then you would die a horrible death pretty quickly. As such the combat systems use an approach similar to that found in Halo 3 where it balances which enemy attacks you based on how interesting that attack will be in the context of combat.

This requires an action selection utility function that is calculates how interesting it would be for that machine to attack. This is based on the current machines state, whether the player is aware of that machine in proximity, how close it is to the player and the amount of damage it has received and dished out. This becomes a precondition of the HTN planning system, and aims not only to provide some challenge, but also to inject some variety into the combat.

Perhaps more critically, when the HTN planner is asked to generate actions for attack groups, while an attack is being selected the other machines in the group are also being given movement behaviours to circle the player and wait their turn. This is done deliberately to create opportunities for the player to exploit, either attacking a passive machine or countering the one that just attacked you. In time you’ll slowly be able to wear them down, largely because the combat system is deliberately leaving itself open to attack.

Closing

As we’ve seen throughout this piece, the machines of Horizon Zero Dawn can prove lethal either on their own or as part of a pack. Through Hierarchical Task Network planning and an intelligent agent hierarchy system, machines can roam the wilds on their own or protect and support one another where necessary. Creating a fun and exciting world to interact with that keeps players on their toes at all times.

In part 2 I’m going to dive into the tools and systems used by individual AI characters: the sensor systems used to recognise and attack nearby threats, how the AI systems need to work in-tandem with animation to ensure these machines look both dynamic but realistic, the navigation systems that allow machines of different sizes to move through the same terrain plus how flying machines such as the Stormbird use an entirely unique navigation system that allows them to explore the world and hunt you down as best as possible.

References