Featured Blog | This community-written post highlights the best of what the game industry has to offer. Read more like it on the Game Developer Blogs or learn how to Submit Your Own Blog Post
In this final part on the AI of the Division 2, I chat with lead AI programmer Philip Dunstan about the secret AI testers that help playtest the Division 2 and the story behind their creation.
AI and Games is a crowdfunded YouTube series that explores research and applications of artificial intelligence in video games. You can support this work by visiting my Patreon page.
In collaboration with Ubisoft and Massive Entertainment, I present three blogs exploring the AI behind Tom Clancy's The Division 2, including excerpts from my interview with the Lead AI Programmer of the franchise, Philip Dunstan.
Part 1 of this series, where we discuss enemy AI design can be found here.
Meanwhile part 2, which explores open-world and systemic design can be found here.
Building a live-service game such as Tom Clancy's The Division, comes with all sorts of challenges. Ensuring the game is stable for players in a variety of online connections, handling the different ways players move through the world and exploring interactive systems and gameplay challenges, but more critically, checking that the game plays as expected so that players aren't getting frustrated because world events don't trigger or missions don't register as complete. While this is most certainly a challenge for quality assurance and testing teams, as the scale and complexity of these games increase, the workloads of QA teams explode in scale. And that's where artificial intelligence can help not just create in-game functionality, but change how the game is being developed.
In this final blog on the AI of The Division 2, I'm going to take a look at the secret AI players that playtest Tom Clancy's The Division with insights from Philip Dunstan: the lead AI programmer at developers Massive Entertainment. I'll be looking at the custom AI bots that are deployed to assess specific parts of the game and how the first games post-launch DLC changed how the game would be tested moving into Division 2.
Tom Clancy's The Division has not one, but two types of bots that are used to help test the games of the franchise: the server bots and the client bots. The Server bots - as the name might suggest - run natively on the server and don't interface with the game like a player would. As I'll explain in a minute, these bots behave completely different from real players and are designed to stress-test the limits of the Division servers. Meanwhile the client bots run as if they're playing a build of the game client-side. They assume control of the game instead of the player, adhering to all the same rules as a regular player, to a point that all the in-game systems genuinely believe that this player character is being controlled by a human. They don't have any special modifications that allow them to manipulate or cheat the game and are built to run on any platform, be it PC, Xbox or Playstation. Their job is to test how the game actually works for players: testing the main story missions, wandering the open world and gathering all sorts of performance stats along the way that help give the developers a stronger understanding of how the game will perform for players when they log into Washington.
The demand for these types of tools is ever increasing. As in-game worlds in live-service games continue to increase the number of potential bugs explodes exponentially. Even if you consider both Division games, it's not just the map of Washington DC being larger than Manhattan, but each update to both games not only introduces new content - which might have bugs in it - but it also can change or impact a lot of the existing content in the game - meaning even more bugs because you broke something that was already working. This is only made worse by the reality live service games need to be updated fairly frequently to maintain player engagement and these updates need to work, so the word of mouth continues to be strong. This is a problem that exceeds the capabilities of human testers: as more content is being built and existing content modified, not only does quality control need to be maintained on all the new content, but everything else that already exists in the game. This is thousands upon thousands of play hours and is increasingly difficult to balance. And sometimes, the requirements of testing exceed the number of available staff who can even sit down and play the game...
Philip Dunstan: "As you can imagine we're building servers that host a thousand players, but it's really difficult to get a thousand players to play at the same time. And especially if you want to know if your servers can stay up for a week it's difficult to find a thousand players that can play continuously to test the stability of your server while the game is in such an early stage of development.
As mentioned in a previous blog, the original Tom Clancy's The Division runs with what is known as a Server Bot: it's an AI player that logs into a Division server and plays around in the game. This is being used to test whether or not the games systems are operated as expected. As Philip explained, while the develop team really benefit from this, the actual AI they built was really simple and, well... it cheats a lot.
Philip Dunstan: "So very early on in the Division 1, we had these server bots that would connect to a game, they would... you know they're actually really stupid. They're not trying to mimic player behaviour at all. They just teleport around the world, they find NPCs to kill, they shoot the NPCs and then they teleport off to a different part of the world. And they've got god mode turned so they can't be killed and they just do this continuously and then every now and then they disconnect from the game and they reconnect or they group up into co-op sessions and they disconnect. We're testing our ability to you know group players, to create all the different phases for the players to join and disconnect. And then surprisingly it's extremely performance metrics out of these bots. Their performance metrics actually very closely matches the type of metrics we see in players, even though they're not trying play like a player."
"We had those in the Division, we honestly would not have been able to ship a stable Division 1 or Division 2. I mean Division 1 and Division 2 were both extremely stable games you know considering how many players we had after launch. If you look at this last year type thing, the number of like significant downtime causing issues that we've had has been extremely low. And we're able to do that because we're able to test it to an extent that we're satisfied through an automated method."
While the server bots were conceived from the very beginning, the client bots are a different story altogether and emerged from an interesting problem during the development of the first Division. But not at launch, rather with the second DLC update for the game The Underground.
The Underground opens up a whole new game mode in the Division accessible from the basement of your base of operations: the James A Farley post office building across the street from Madison Square Garden. In the underground players would complete procedurally generated missions comprised of different enemy factions hiding out in the tunnels underneath New York. And this is where it introduces a new problem: unlike the rest of the Division, if a mission is going to be procedurally generated how do you test each possible permutation to know it's going to work?
Philip Dunstan: "The client bots were interesting, the Underground, because it is procedurally generated had a sort of problem which had been unique up until that point. Up to that point we'd be able to test whether a level could be completed by having QC run through that level and see if we can complete it. We have a large test team at Ubisoft that is constantly playing through the levels testing things like 'is this level completable'. And that worked perfect fine for the launch of the Division and survival mode. But for underground, we had you know hundreds and thousands of different variations of the level. It no longer became possible to test this manually. We had a technical problem at the time as well that our navmesh generation wasn't consistent enough, that when we generate the navmesh for the underground level one of the variations might be playable, but later on when someone had moved some props around and we may have had a navmesh break on a subsequent generation. So it became not just impossible from a practical sense of how many testers you need, it just wasn't even feasible at all to manually test."
The Client bots were headed up by one Massive's partner studios working on the Division: Ubisoft Reflections based in the UK. As mentioned earlier, the team opted for the more challenging task of creating an entirely new system. The AI players are not based on the existing AI archetypes, instead it's a custom built AI layer that directly controls the players inputs. This helps keep all the development of these tools isolated from main enemy AI but as mentioned, it means every system in the game still believes that a human player is playing the game. The system was subsequently interfaced into the debug console and tools, allowing for a variety of game actions to bypass the controller layer and be processed by the player character. This means that just like a human, the actions it's trying to execute only work if the current game state permits it.
One of the first priorities for the bots was to the test navigation mesh coverage. Navigation meshes are the data structure baked into a level that tells AI characters where in the world they can move around. Without a working nav mesh, no friendly or enemy AI would be able to walk around the map. Hence if any of it is broken this needs to be identified immediately for designers and programmers to test. In addition, follow bots were built that allowed for AI players to follow human ones, once again checking how AI characters might be able to use the navigation mesh to move through complex environments and combat arenas. Plus simple combat behaviours that - while they didn't pay attention to their health or avoiding any hazards - would eliminate targets simply by turning the in-game camera towards an enemies head and then pulling the trigger.
But in-time this scaled up from the more low-level tests of movement space and simple combat, to being able to take on an entire mission. This requires a lot more complexity and interfacing with the separate mission system built into The Division's codebase given it needs to know what the objectives are at any point in time and naturally these shift throughout a given story mission. This requires a more nuanced process, whereby the bots kill all the enemies in an area, follow the path to the objective marker, destroying specific objects if expected to and triggering any and all interactions it finds within a radius of itself.
With the client bots successfully implemented, they could not only test hundreds of permutations of the Underground missions over a weekend, but they could also run tests overnight on all the main story missions from the first game.
Philip Dunstan: "And this was y'know, really successful for underground, it became an important part of our tool for walkthroughs, but also even more importantly I think leading into Division 2 it became even more successful as a means to test our client performance through a level. You could have a whole suite of consoles playing through the level and recording their FPS and memory usage every night or 24/7 and creating reports from that. And same for the open world, we could have a group of client bots moving through the open world and finding parts of the world where performance becomes a problem. So they became like a really important part of console performance testing and they're still being used like that on Division 2."
Transitioning this technology from Division 1 to Division 2 still required a lot of extra work, as well as a change in workflow for both testers and the level and gameplay designers. The progression of missions in the Division 2 is not as linear as it was in the first game, hence the bots might become confused in certain parts of the mission as to how to proceed. So throughout development on Division 2, invisible testing markup is laced throughout each mission of the game by the level designers. It's completely invisible to players and has zero impact on our experience of the game, but the client bots can read this mark-up as directions for continuing their work. Mission tests are ran nightly and help ensure errors don't creep through into production as they identify mission content failing or game systems not executing as expected. There is also a separate error type for logging when the client bot wasn't working as expected and the developers strive to ensure all three of these categories have zero failures in them at all times.
In addition, the open-world testing is used by the QA and Tech Art teams on the game to identify areas of the world where there are performance overheads and they report specific bugs once they've looked at the data. As the client bots visit all playable space within the world, they identify areas of the world they get stuck in and cannot return from as well as framerate drops that could be due to high poly counts or issues with textures or particle systems in the environment.Lastly, there is also the AI Gym: which is dedicated to testing both the bots functionality itself, but also core gameplay mechanics should changes be made.
There are still limitations to what these systems can do, largely in part due to the complexities of the game world that would seem intuitive to a player, but the client bot might need some extra hand holding. And of course despite this big push into automation, there's still a lot of value gained from having people sit down and play the game as well.
Philip Dunstan: "There's definite restrictions to what our client bots can do. Again they're not really trying to play like a human, we're not trying to model human play. They move through the levels on a 'golden path', a hand-placed level-designed path on this is how you move through the level. They need to know how to interact with the doors or they need to know how to interact with the laptops to unlock the next part of the level. So they do require some manual scripted setup. So they're not really playing like a player would play. But they still provide a lot of benefit even with those restrictions.
You still need to have dev testers y'know, testing that you can't walk off, there aren't nav mesh blockers preventing you from getting to parts of levels because the client bot will move along the golden path and not check the areas of the combat space. But you get an early sort of smoke-test system of saying y'know 'is there something significantly wrong with this level'?"
As games continue to become larger and more complex, there is a real need to automate critical aspects of development. Be it testing frameworks, batch processes of art pipelines, animation controllers, design tools and more, all of it serves the needs of the development team. Allowing for programmers, artists and designers to focus their efforts on delivering the best game they can. Artificial Intelligence is slowly changing the way in which video games are built and is being applied in ways that players would never really think about. The real achievement is by employing AI in these new and pragmatic ways, it helps keeps the problems that can emerge in game development manageable. It keep projects on track and in budget. But it's important that players understand the challenges of how games are made: be it the Division 2, other Ubisoft projects or the industry as a whole.
Special thanks to Ubisoft for the opportunity to work with them on this project. And of course to my patrons who crowdfund the AI and Games series.
Read more about:
Featured BlogsYou May Also Like
.png?width=300&auto=webp&quality=80&disable=upscale)
