Analyzing the AI of Super Mario 64

'AI and Games' is a crowdfunded YouTube series that explores research and applications of artificial intelligence in video games.  You can support this work by visiting my Patreon page.

Super Mario 64, it’s one of the most important and beloved videogames ever made. It set the standard 3D games at a critical phase in the industry. Not just influencing titles on the Nintendo 64, but subsequent releases on many a competing platform.

But how did it actually work? Is there anything interesting happening in the minds of the Goombas, Koopas and other enemies we find idling around Bob-Omb mountain, Whomp’s Fortress, and Tick Tock Clock? Well, let’s find out.

Cracking open the Hood

Super Mario 64 is now over 25 years old, and while the AI may be quite simple, it’s important to recognise what this game represents. It’s a pivotal point in the history of game development. It is the top-selling title of the Nintendo 64 and is reflective of a time when game development was migrating to 3D. It had a huge influence on how three-dimensional games were designed. While it wasn’t the first 3D platformer, nor even the first 3D title made by Nintendo, Super Mario 64‘s legacy cannot be understated, with designers such as Tim Schaeffer and Michael John outright stating the huge influence it had on projects of their own such as Psychonauts and Spyro the Dragon respectively. While areas such as camera controls would merit further iteration, aspects of visual design, character animation, player movement, level and mission construction set the template for many other games to follow.

While the AI is undoubtedly rudimentary, it’s worth analysing to see how Nintendo went about structuring and designing behaviour. To make this happen, I downloaded the Mario 64 decompilation hosted on GitHub. This project was first published back in 2019 and is just one of a number of works from a community of hobbyist decompilers who take it upon themselves to reverse engineer the raw binary of the ROM file into parsable, legible code in the C programming language. So what I’m covering is based not on the original source by Nintendo themselves, but the reversed engineered code that is publically accessible and designed to be as close to the original as possible.

This decompilation isn’t driven by a desire to steal Mario assets or make new ports of the game. Rather, it was a mechanism to help speedrunners find new exploits. For many of the developers involved, being able to piece together the puzzle, one function at a time is the real reward. Decompiling source code like this isn’t an easy task at the best of times. But it’s made all the more difficult by having to figure out just what development tools were being used when the game was built back in 1996. To decompile a game like this one, you need to have an idea of which version of the Silicon Graphics IDO compiler was used to generate the original ROM file. And testing all of this requires emulating the system behaviour of the original Silicon Graphics development units: the SGI Visual Workstation, given this is what would have been used by developers building games for the Nintendo 64. It’s a real feat of engineering. If you want to learn more about this work, including the efforts to decompile the likes of The Legend of Zelda: Ocarina of Time, check out the arstechnica article on the subject.

The decompiled source code, once properly configured, allows you to compile the ROM file for each regional version of the final game (Japan, US and Europe). However, in order to create a fully playable game, you still need a copy of the original game ROM, given this project only contains community-written source code. It doesn’t have any of the in-game assets. Meaning you can’t just compile it and voila: free copy of the game.

Objects & Behaviours

The first thing you need to understand about how Mario 64’s code is broken up, is that pretty much every single element in the world, outside of the level geometry, is considered an object. And then most game objects can then have a behaviour attached to them as well.

Behaviours enable a variety of common functions, be it to spawn an object at the location such as a coin, make the object fall as a result of an interaction, customise the hitbox for collisions between in-game objects, turn to face the camera when prompted, disable the renderer because its destroyed and much more. Behaviour scripts are not exclusive to AI characters: moving platforms, doors, coins, fireballs, shockwaves, and even some trees and butterflies are considered to have behaviours, given they all respond to the game world in some form or another as a result of interactions or the passing of time.

But all objects that carry Behaviour Scripts have two main functions, their begin() function and their update() function. These do two very specific things, begin() is responsible for configuring this object for the game now that it has spawned in, while update() handles the behaviour of the object for each tick or frame of the game. Mario 64 typically ran it around 30 frames per second, with some of the most intense parts of the game running at around 20 frames. So while begin() is only called the first frame the object exists, update() is called every frame afterwards. If you’re a game developer yourself, then this will all sound very familiar to you, given the likes of Unity and Unreal engine have a similar structure.

The update() function handled the execution of common behaviour commands for each object type, unit movement along the x and z-axis, as well as on the y-axis, compensating for gravity. Plus it pays attention to all the logic for managing timers for actions, whether the object had switched from one action to another, all of which is a crudely implemented finite state machine, largely derived from conditional branches in the code. But the most important thing, given it’s literally the first bit of computation each object does, is checks where Mario is relative to it.

The first two calculations of each object with a behaviour script are to figure out the distance from Mario and the angle required to turn and face him. This is used for a variety of AI features, which we’ll see shortly. But it’s also used critically for the rendering of in-game objects. Every behaviour-driven object in Super Mario 64 decides based on its proximity to Mario whether it should render or not. Each has its own drawing distance parameter, and every frame it checks to see if Mario is now farther away than the draw distance, at which point it tells the renderer for that object to disable itself. Interestingly I couldn’t find evidence to suggest that the behaviours don’t execute when Mario is too far away.

The one exception to this is if it’s in a room. Mario levels are built either as open spaces, such as Bob-Omb Battlefield, or they’re comprised of multiple rooms. The three notable examples of this are of course Peach’s Castle, but also Big Boo’s Haunt and Hazy Maze Cave. In these instances, the object will only render when Mario is in the same room. And in fact, a lot of behaviour scripts are also only activated when Mario is in that room as well.

Now outside of the main behaviour functions, an object would also carry a pointer to a separate behaviour command script. These scripts are written for more bespoke elements of the game, and allow for the execution of the bulk of the game’s interactions and behavioural logic. There are over 500 of these in the game, and range from everything like specific NPC behaviours, to the triggering of red coin collections, activating traps, even ending the level by collecting the power star.

But on top of all this, there are hitboxes that dictate how Mario interacts with objects If you’re not familiar with the idea, the hitbox helps tell the game logic that the two game objects are interacting and with a bit of logic, we can make it so that the depending on the context, different stuff happens. Like Mario killing an enemy, or the enemy hurting the player. Interestingly, pretty much all enemies in Mario 64 have two hitboxes and they’re both cylindrical. The standard object hitbox for a lot of in-game collisions and logic, but also the hurtbox, which is exclusively for situations where Mario will get hurt. Most NPCs use the hurtbox, and it is often a different height and radius compared to the regular one. However, there are some exceptions to this, notably Koopa the Quick, who doesn’t use the hurtbox given he just pushes by you rather than actually triggering damage upon collision.

NPC Design

Like the main behaviour script, the separate behaviour actions used by NPCs have their own begin() and update() functions. I’m going to walk through some of the interesting stuff that I found in there.

As mentioned already, all game objects compute their distance and angle relative to Mario for rendering, but also many of the NPC use this for their core logic. Those who have played the game will have noticed this, given the likes of Koopas and Goombas react to Mario’s presence.

But on top of this, many of them record their ‘home‘. This is a location in the game world that is, meaning they store a reference to where they spawned into the map, or very close to it. This is used so that the NPC knows roughly where in the world it appeared when the level loaded in.

It’s worth mentioning that NPCs in Mario 64 can’t compute on the likes of a navigation mesh to figure out how to move around the world. Because, well, there isn’t one – and what we now consider a navmesh in game engines didn’t exist until Quake III Arena in 1999. Instead, NPCs such as the Goomba and Koopa Troopa will wander around, with fixed rules on how far they go before changing distance and often rotating in fixed angles. You’ll note that they stay on the surface they started on the majority of the time, and a lot of Mario 64’s levels aren’t filled with clutter, meaning it’s harder for them to get stuck behind obstacles. There’s only a handful of enemies that will ever move across platforms but that’s largely due to dumb luck from jumping, or they fly around than anything happening at a code level.

But as they move around, they often pay attention to two things. The first is whether they’re going to collide with walls, in which case they will turn away from them. But how does it know it’s facing a wall? There are functions that allow for them to resolve collisions based on their current heading, and often there’s some wriggle room there. The same applies for when they are looking at Mario, given it’s not based on their exact heading (given that’s too precise), but whether within some range of their heading that the object is in front of them. In some instances, notably the Goomba, a character will simply react to Mario being within a certain distance of them, regardless of heading.

The second key fact they focus on is how far away from home they are. Each NPC has different rules on how far away from home they’re allowed to go, and whether it’s the total distance from home (which is the logic used by a Goomba and Koopa) or if the distance on a specific axis has been reached (which is used by Bob-Omb) or in the case of the Boo, whether it falls within a radius of their origin (calculated using their X and Z movement). In any case, the general rule is that they will turn around, face towards home and start wandering back in that direction. But in some cases, notably Bob-Omb and the Bully, they only return back home if it’s deemed safe, which is determined by whether Mario is currently in proximity of their home location.

And it’s this logic that encapsulates most characters, but they all have their own unique caveats. Goombas make hard turns away from walls and double back to home if their total distance is too far away. Then they will chase Mario if he’s too close, regardless of angle, at around 500 units. With chases always pre-empted by a jump. For reference: distance units almost map directly to in-world centimetres. Given Mario is 161 units tall in the game, and is also – according to the Mario wiki – 155cm tall. So when we say within 500 units for the Goomba, it’s just shy of 5 meters.

Meanwhile, Koopa Troopas have a similar logic, albeit they run away from Mario within 300 units. The interesting part in the Koopa logic is that once you jump on them, they can lose their shell. In general, they will try to run back to their shell, but quite often Mario is too close and forces them to run away. However, if the shell is close enough, despite Mario also being close, it will take the gamble and try and dive back into its shell.

As mentioned Bob-Ombs patrol based on how far away on any axis they’ve travelled from their home. Although to be fair it’s difficult for them to ever go anywhere of distance without subsequently blowing themselves up. But unlike enemies such as Goombas, when a bob-omb dies, it triggers a respawner in the code, that will create a brand new bob-omb where the last one appeared after Mario has safely left the area.

Meanwhile, other enemies like Whomps check their distance from home and compares that against a predefined patrol distance. This is then configured on a per-level basis. In the case of Whomp, it has the same patrol distance for all levels, with exception of Bowser in the Sky, which is much much shorter.

Heave-Ho’s that appear in Tick Tock Clock and Wet Dry World also exploit the home and Mario locations, but in a slightly different way: in this case the Heave Ho records its home location on start. But once Mario gets within a certain range of home, then it starts homing in on you.

Now I can’t possibly talk about Mario 64 and not mention the fish! The fish spawn in two variations, blue and cyan. Plus the schools of fish that spawn come in two variations: the large school of 20 fish, and the smaller school of 5 fish. Fish when spawned in start with different starting speeds (given a little bit of random noise is added), plus their height in the water is also varied to make sure they’re not all clumped too close together.

The fish themselves have a pretty simple set of behaviour, they dive and ascend, turn a fixed angle plus some random noise, and then make sure they turn back. Their height is typically capped such that they can’t get close to the water surface, with a strong negative vertical speed applied if they get too close. The one exception to this is the Secret Aquarium because there is no water surface in that level.

To make sure they don’t wade too far from their origin, fish can only move within a fixed range of around 700 units. But also the school usually spawns somewhere between 1500 and 2000 units of Mario being in the map. So, 700 units mean the fish only really swim in a range of just shy of 7 meters. And the fish spawn around 15-20 meters away from Mario. Plus, all fish will react to Mario being in the water. When Mario is less than 150 units away, they turn to be pointing away from Mario, and all of them speed up. Setting a standard that the games industry would continue to follow, for many years to come.

For more quirky facts, be sure to check out the ‘speed round’ segment of the YouTube video.

Closing

Given this was the 64th episode of the YouTube series, it made sense to make this all about Super Mario 64. For more Nintendo 64 analysis, consider my writings on the AI of Goldeneye 007. Plus, if you enjoy these episodes where I crack open the source code, be sure to check out my work on the AI of Command and Conquer, Half-Life and FEAR.