This post, written by Sean Packham, originally appeared on Kythera AI's website in October 2020.

Aquanox Deep Descent is drenched in atmosphere and backstory: navigate dark and eerie routes through fallen cities, discover vast caverns bathed in dystopian palettes and bursts of color from ocean plants, wrecks, nano-plankton and the fireworks from head-to-head combat. Digital Arrow has created a dangerous world where relationships between powerful factions are balanced on a knife edge. This tension is felt in the excellently voiced dialogue, but more importantly, throughout the game's core combat loop – a rare feat to achieve.

Kythera AI worked closely with Digital Arrow and THQ Nordic to bring their vision for a worthy addition to the Aquanox franchise to life. Digital Arrow's team used a number of our features to build engaging AI in Unreal Engine 4. They made extensive use of two Kythera AI systems: our advanced behavior system, and our 3D navigation (pathfinding and avoidance). This post is a behind-the-scenes look into what it takes to build rich and engaging AI-driven gameplay for UE4. 

Building believable behaviors

Norbert Varga – Creative Director

Digital Arrow’s vision for AI was to create an experience where players could learn to identify different enemy behaviors and develop their own effective countermeasures. “We want the player to feel they can effectively kill an AI if they figure out how they are reacting,” said Norbert. “This has created an experience some have described being 'Souls-like' – identifying and countering behaviors.”

Earlier in the game’s development, the AI swapped between different types of short and long range weapons, but people found it confusing to identify and counter these strategies. Now, the AI gets a weapon loadout which defines its behavior. “When we swapped to the loadout system, AI behavior became more readable, allowing us to make them more dangerous,” said Norbert. Now, If you read the AI incorrectly you can die quickly if you are not careful. Ships with similar loadouts were grouped into three types. There are up-in-your-face fast and short range AI, mid and long range AI which take strong cover to fire torpedoes and shards (rail guns) at a distance, and finally AI mini-bosses. These bosses have identifiable appearances and more dangerous weapon combinations or secondary weapons like body turrets. “By layering the AI this way, we created an immersive multi-squad behavior,” said Norbert.

The player is given some extra chances to adapt to incorrectly identifying enemy behavior or when ambushed. AI have an accuracy wind-up, so their first attacks will often be near misses to alert players. The game has no instant-hit weapons, so players can always dash to safety. The AI behavior also shifts in accordance with player health to increase the intensity of the battle and the sense of achievement when players survive. 

Kythera AI’s Cover system identifies suitable cover spots for AI ships to consider. Orange = too close to a hostile. Red = can be seen by the enemy. Green line = passed all the preceding checks, now going to check if we can pop out of cover. Yellow = can't pop out from cover and see target. Green = good candidate for cover.

That being said, the player and AI can still rapidly destroy one another. This is why we love the cover system. It allows the AI to search the environment for suitable spots to seek shelter, recuperate or launch a counter attack. Norbert says, “If you see the AI using cover, you should be taking cover too!” This behavior becomes predictable, forcing the player to react quickly: either chase them down so they don’t get cover, or end them. The cover system was created by marking up levels with cover areas which AI could search for and navigate to.

All the behaviors for Aquanox Deep Descent were created using Kythera AI Behavior Trees and State Machines, and custom C++ interfaces to expose ship abilities to behavior trees. These tools allowed their developers, designers, scripters and testers to easily build and debug AI behaviors from our behavior tree editor. Kythera AI Behavior Trees are similar to Unreal Engine behavior trees but are engine-agnostic and allow more sophisticated control flow, including a powerful State Machine system to control behavior tree switching.

View fullsize

Many of the AI innovations in Aquanox Deep Descent paved the way for more advanced features in new versions of Kythera AI. We built our Autocover system that automatically marks up 2D navmeshes with suitable cover points. To make it easier to query the environment for suitable points of interests based on size and distance, we built our Spatial Query System (SQS) – a more advanced version of Unreal Engine’s EQS which integrates with Autocover and allows developers, designers and scripters to write queries in C++, behavior trees or our custom DSL. The combination of our core AI, Behavior Trees, Autocover and SQS is the most advanced set of AI tools on the market. 

Navigating the depths

Norbert Varga – Creative Director

Navigation is often something taken for granted in modern games but their expansive and dynamic worlds pose new challenges. Players expect characters to flawlessly find paths through complex and often dynamic environments and avoid other entities along that path. When it just works you can get on with enjoying the gameplay, but when it doesn’t it sticks out. 

Unreal Engine 4 didn’t, and still doesn’t, have a 3D pathfinding solution, so the Kythera AI team built the Octree 3D Pathfinding system for Digital Arrow. This solution automatically divides the world into a tree of ever-smaller voxels to allow AI ships to find paths and travel through complex tunnel systems, navigate to suitable cover points and more. “The AI movement eases in and out of turns and accelerations to navigate paths, creating smooth and realistic acceleration and deceleration curves,” said Norbert. It is an essential feature to turn rigid algorithmic paths into believable behavior.

To make sure pathfinding was fast and efficient we implemented our Navseeds feature, which allows the designer to place one or more markers in the UE4 Editor that indicate regions that they would like to be navigable. Kythera AI then finds all navigable portions of navmesh that are not reachable from any of these markers and removes these from the navmesh. This reduces the memory footprint of the octree and improves performance.

Kythera AI Octree 3D Pathfinding for Unreal Engine. The octree is shown in red and the path in white, extending from the top left of the screen into the distance.

“The realistic and predictable acceleration and deceleration also contributes to the gameplay through aim prediction. If you see an AI ship strafing to the right you know to expect a certain amount of deceleration and can predict where to shoot,” said Norbert. The AI has a realistic perception and detection system. “If you target lock an AI, their sensors will warn them of an incoming projectile,” said Norbert, “they have the exact same tools the player has.” Players need to choose between target locking or approaching the enemy’s blind side. 

AI will also perform less flashy dynamic object avoidance while navigating paths or following scripted commands. There are a number of approaches to solve avoidance problems, either through updating the Octree when the environment changes or steering around dynamic entities. We opted for robust ORCA-based dynamic steering in Aquanox. Inspired by the project, we’re also adding full Octree regeneration to Kythera AI to allow the environment itself to change during gameplay.

AI need to be able to avoid large dynamic obstacles, like jump gates and the Tupilaq, along their paths to objectives.

It’s been an exciting journey working with everyone at THQ Nordic and Digital Arrow to create a unique gaming experience. We look forward to building more games with them.

Experience the atmosphere, story and AI-driven gameplay for yourself: Aquanox Deep Descent is now available on Steam, and features a gripping single-player and 4 player co-op campaign as well as online deathmatch.

Interested in using Kythera AI to build unique AI-driven gameplay for your studio’s next title? Contact us to find out how we can help your team move faster with our complete AI middleware solution for UE4 and bespoke AI development and support.