Postmortem: Crafting AI for a symmetric asynchronous multiplayer game mode of CastleStorm

What is CastleStorm

CastleStorm is a mixture of 2D physics destruction and tower defense brawler, developed by Zen Studios. It has been released for Xbox 360 in May 2013, followed by the releases for Steam, PlayStation 3, PlayStation Vita, PlayStation 4, Nintendo Wii U, and Xbox One. Its free to play version for mobile was launched a year later, in May 2014.

The core gameplay offers several different ways for the players to achieve their goals.

They can send out troops which will fight on the battlefield. There are nine different units for each of the four nations available in the game. Some of them just walk and smash everything, others fire arrows or cannonballs, some can heal or buff the troops around them.

Firing from the ballista aiming the troops or the castles is another way to take control of the battle. There is a wide variety of available projectiles, most of them with special abilities, like splitting to multiple other projectiles, exploding on impact, converting, transforming, freezing or healing the unit they slam into.

Casting offensive or defensive magical spells, such as damaging enemy or healing friendly troops, summoning other non-spawnable units or protecting the battlefield with the projectile shield is an option as well. Cloning key units or turning the enemy troops around on the field may really come in handy, too.

The players can also summon a hero and jump into the battlefield themselves in hack ‘n’ slash style to dominate the enemy troops. This can turn the tide of a battle sometimes.

The castles are constructed from troop rooms, which determines what units the players will have on the field. It also consists of different support rooms that can grow the number of troops the players can spawn, or speed up food production, increase defenses against enemy projectiles etc.

Introduction to multiplayer

We've all played classic asymmetric asynchronous games on our mobiles, where a player sets up his defenses and leaves the base, another player then attacks him and coordinates their troops in real-time. These games require little to no AI, as simple as path finding and basic targeting, but that's all.

However, when we've come to the point that CastleStorm - Free to Siege should receive an update with a completely new game mode, we've had several questions for ourselves, and there were no other titles on the market to look at for reference. The biggest difference between CastleStorm and the other async games is the fact, that CastleStorm's gameplay is all symmetric. The game on consoles and Steam has several different multiplayer game modes, but the standard versus is based around two major objectives: capturing the enemy's flag or destroying his castle. We didn't want to change the core elements of the gameplay, so bringing the standard roles to the game was impossible, where the defender would put up defences, and the attacker would simply send out troops…

Preparations

Our first step was to create an actual AI for the opponent. As the base game is built on scripted spawn system, occasionally firing ballista, and the magic casting and hero summoning ability only allowed for the player, they were fundamentals we had to set first. The adaptive AI system created for the multiplayer is strictly data-driven, meaning that it has no pre-programmed behavior to follow, everything comes from - in our case - config files.

How it works

Being an event-based system, it can be split into two major components. On one hand, several different types of events are generated while playing the game (like a unit is spawned, a projectile is fired, a magic is cast, a unit is damaged etc.). If an event occurs, the system looks for an action, which will define what reaction should the AI take in such a case.

First, it runs through a large set of filters to see if this event is one it should respond to. The filters are meant to be able to fine-tune every event to match exactly what we are looking for (like whether a unit/spell/projectile belongs to the player or the opponent; is it a flag carrier; has more/less hp than some ratio; is closer to the opponent's gate; is a specific type; has a number of other units around etc.).

If a match is found, the system looks up the action to see what to do. Each action can contain one or more reactions, like firing a projectile, casting a magic, sending out a set of troops, or using the duck ability. The AI checks every reaction if they are available (is the projectile not on cooldown, or is there enough food for the unit to spawn) keeping the preset order, and if it finds one, it is added to its queue.

The AI operates on four different layers: - Ballista mode for firing projectiles - Magic mode for casting spells - Troop mode for sending out units - Hero mode, for playing the hero

Each layer has its own queue of responses, each with the desired projectile/magic/troop, the target and the delay after it should be executed. Whenever the delay reaches zero, the response is executed and removed from the queue. If there are no responses in the queue, no action is taken.

But being an asynchronous multiplayer game, the player should have the option to customize his own defending AI. Building the castle and setting the desired equipment is one thing, but if all defenders would use one preconfigured AI, it would ruin the fun after a very short time. That's why we've came up with an idea: the AI should be able to learn and adapt the playstyle of the player, and use the patterns against his attackers while being offline. This way, if a player attacks two opponents with the same equipment and castle, the game will be played out differently, as the two players probably don’t play exactly the same way.

The adaptive system

To achieve this, the system monitors every action the player takes, and runs through the latest events to find the most suitable. If none is found, it generates one from the current snapshot of the battlefield. At the end of the battle, a human readable config file is generated (similar to what we've created by hand), containing every event and action the system has generated based on the players session. The next time we attack this player, this newly generated file will be used for the AI. It is important that the system doesn't overwrite data, it extends the previously created datasets. If a new event occurs and an action is associated to it, it will look up the previous ones to check if there is no duplication, to avoid redundancy. If the event has already been recorded, the system simply adds the action as another reaction to that event. It is also possible for a player to change his game style over time, as the system also monitors what troops he sends out the most, and will prefer the ones with the highest usage, so switching to new tactics while keeping the same equipment will be followed by the AI as well. For example, if the player prefers to use golem-priest combos for the first ten battles, then switches to golem-catapult later on, after a few battles the AI will start to prefer the latter too.

Several other factors are also recorded, like accuracy and reaction times. When we were getting closer to the open beta, we realised that the same config would produce the same gameplay on lvl1 and lvl20, and thus would not provide enough challenge at higher levels. We've decided to tune the limits of these values based on the players level, meaning that on higher levels if the player is accurate and fast, the AI will be able to catch up and operate on the same degree, but on lower levels it is a bit handicapped. The end results were quite satisfying, as the same config given to a low level AI would produce a good challenge on the same level, and after upgrading to higher equipments the battle became much more intense, but kept the same difficulty.

This adaptive system proved to be a quick learner. After only 4-5 battles it can already produce a customized AI which will be much more efficient compared to the default that everyone starts with. Looking at what we've been trying to achieve, the goal was similar to getting an ever-learning neural network, while eliminating the need for millions of iterations, and having to freeze the system once the correct weights are found. We needed a continuously changing system that can achieve good results from as few samples as possible.

When we ran our tests it turned out that such an AI data file can be more than 200 Kbytes in size. This is not much space for a single user, but preparing for heavy loads and keeping in mind that the players will download this data every time they start a battle, we knew that we should cut the size as much as possible. The first step was to convert the human readable text file to binary, which cut the size to half. After this was done, we've added our custom compression to the stream, being able to reduce the size of any data file below 10 Kbytes (usually between 4 and 6 Kbytes).

Closing thoughts

As the new game mode has just been launched on both the Android and iOS platforms, we are excited to see where it will go and how the players will find its overall value. It was a great experience creating this new mode and experimenting with different ideas. Crafting adaptive systems is always difficult, as balancing is not so trivial. However, when everything fits together, it’s nice to see something different every time.

Check out the game for yourself:

Google Play: https://play.google.com/store/apps/details?id=com.zenstudios.castlestorm
iTunes: https://itunes.apple.com/us/app/castlestorm-free-to-siege/id726232588

Original post: http://www.bondorandras.com/postmortem-castlestorm-adaptive-ai-system/