Implementing a replay system in Unity and how I'd do it differently next time

While working on Sportsball I decided it would be a great feature to be able to show replays of matches. The instant replay feature is pretty popular not only in sports, but also competitive video games like Towerfall. Even 0Space has it, although by playing that game I realized that it’s very easy to make a replay system that doesn’t actually work. Although hilarious on occasion, as a player, I want to make sure you can show the world that amazing box pivot score I did to take the game at the last moment. Here’s how I went about implementing a deterministic replay solution in Unity version 4.2 and 4.3, requiring only recording and replaying the player’s inputs.
 

STEP 1: RESEARCH

Since I’ve never implemented a replay system before, and it’s something that many games have done, I figured I’d learn from past examples. My main source of information came from this Gamasutra article. It gave me a good overview of the two main approaches to building a replay system: saved state and deterministic.
 

Saved state - Save the (important) data of all objects each frame. Replays are played back by restoring the state (of variables) of recorded objects.

Pros:

Cons:

Deterministic - You save only the initial state of all objects, then player inputs each frame. Replays are played back by restoring the state of the game’s input each frame.

Pros:

Cons:

I decided to go with a deterministic solution as I was planning on targeting consoles with Sportsball (with us eventually settling on the Wii U, which has 1GB of memory available). Later on I watched a talk by Jonathan Blow, discussing the time rewinding mechanic in Braid. To my surprise, he implemented a saved state approach. I figured that would be far more challenging due to the fact that the game could record hours of gameplay on an XBox 360. He decided to embrace the challenge of optimizing the data saved, instead of fighting with maintaining a deterministic game. This was also the approach of The Bridge as I learned after talking with its developer Ty Taylor. The systems they implemented were great, but not ideal for Sportsball. With up to 80 objects being recorded each frame, it may have been a larger challenge optimizing that aspect of the system. You’ll have to make a call for your own game.
 

I also want to note that the Unity asset store has EZ Replay Manager for saving replays. After using their demo, I didn’t like that the game appeared to play back in a lower framerate. Maybe I could have gotten the replays running at 60FPS with enough effort. But, I decided to create my own because I was interested in learning the system inside and out.

STEP 2: NON DETERMINISM IN UNITY
 

For the most part, making a deterministic game shouldn’t be too difficult these days. With the IEEE standard for floating point numbers, you can (mostly) ensure that even with floats your simulation will run the same on a single machine. But, in short, you need to make sure that with a given set of inputs, your simulation will do the exact same thing every time you run it. Unfortunately, Unity has some issues doing this out of the box.
 

Random.Range(): This function is great in that every time you run your game, it will give you a different random number. But, that’s not what we want. I need to ensure that a random number I pick will be the same random number each time. Unity has a Random.seed variable, which will ensure it uses the same seed for calculating a random number. But, even this wasn’t enough for me. I discovered that if multiple objects accessed Random.Range() with a seed set at the beginning, those future random numbers were not guaranteed to be deterministic. So I made a more brute force approach. I created a public static function that allowed me to get a deterministic random number:
 

public static int GetDeterminedRandomNumber(int min, int max){

       UnityEngine.Random.seed = staticSeedNumber;

       int num =  UnityEngine.Random.Range(min, max);

       UnityEngine.Random.seed = staticSeedNumber;

       return num;

   }

This gets me a random value, but then resets the seed for this frame. I can’t be positive on this (so please correct me if I’m wrong) but I think Unity randomizes the seed after each use of Random.Range(), resulting in non deterministic results, even if you set the seed. So before each Update frame in my game, I set the staticSeedNumber and, through the use of GetDeterminedRandomNumber() I make sure to use that same seed for the whole frame. For Sportsball, the staticSeedNumber is the time in milliseconds that the match was started subtracted by how long in milliseconds the match has been running. This means that each frame of the replay has a unique seed. This was also an easy way for me to get a unique seed for each replay, as I was already storing when the matches start.

EDIT: As noted to me by Reddit user Dest123, this is not a very helpful function! It will always returns the same number given the same range. Instead, this function is not necessary. The seed should be set at the beginning of the replay, and every call to Random.Range() MUST happen in the same order. More details on the reddit post here.

OnCollision(): Through my experiments I discovered that Unity’s built in collision system is not deterministic. I would have collisions happen on one frame or another. Normally they are very close (within a frame) but not exact. Because Unity’s raycasts also use this collision system, I was out of luck.
 

This is probably the largest challenge of implementing a deterministic solution in Unity: you have to program your own collision from scratch. Fortunately, there is tons of documentation on programming collision. It was definitely a struggle for me, as math isn’t my strong suit. But, I was able to build a solid collision system with Miguel Gomez's article on Simple Intersection Tests for Games. And if you’re like me and not super familiar with linear algebra, I suggest checking Wildbunny’s Vector Maths primer. There are many articles on collision, but since Sportsball is a precise 2D competitive game, I liked Miguel’s approach the most. His collision implementation, by using sweep tests, has the ability to predict collisions before they occur. By detecting collisions from sweeping the potential area of contact, we can prevent high speed objects from skipping over collisions when moving too fast.
 

Once you’ve converted the math over, you need to actually decide when to perform your collisions. I used the following loop:
 

void Update(){

       GetPlayerInput(); // this takes player actions from wherever it arrives from (controllers, saved replay data, or network packets). This will call functions on the objects in the scene that should move. For example, if the player presses jump, this is where the player object will be told to set its velocity to what it should be if jumping.

       CheckCollisions(); // based off the input from the player, we can determine now what we want to allow. For example, if the player presses the jump button, which sets the player’s velocity to a very high number in y, we will perform a collision check based off of the desired velocity. In this way, we can check if the player will collide against anything and set the player’s velocity so that it will move flush to the colliding object, as opposed to going through it and having to correct next frame.

       object.UpdateAt(); // this is where we actually apply the object’s new velocity calculated after limiting it due to any collisions.

   }
 

To reiterate, each update we take the player’s intention, we limit the ability for the player to move based on any possible collisions, and then we perform the results of the interactions. By using this loop, we can ensure that objects don’t pass through each other, even at high velocities (if you implemented a sweeping collision check).
 

Time.deltaTime: Many games prefer to be run framerate independent. This means that no matter how fast or slow your computer is running, the simulation always updates at the same pace. (If you’re unfamiliar with the concept, there is a pretty good discussion on Stack Exchange.) Since we need each update loop to perform the same way every time, we can’t rely on Time.deltaTime, because we don’t know that a frame will take the same amount of time to process the next time we run the game (in fact it almost certainly won’t be the same time). My solution was to lock the game simulation to the frame rate. So, for my game, every object that needs to be recorded implements an UpdateAt(int deltaTime) function. I pass through an amount of milliseconds I want to pass in that function when I call update, which allows me to guarantee that the simulation updates consistently each time. You’ll want to set this number based on how long your frames are, and vary it to adjust the speed at which the objects in your game world move.
 

As a note here, you could go ahead and pass Time.deltaTime through the UpdateAt() function if you wanted to. You’d have to record what the deltaTime for that update loop was each frame, so you could use the exact same time to update your objects during the replay (which you should do anyway to record time stretching effects). I didn’t like this implementation because when replaying the game, it would feel like the game was sometimes playing faster or slower, and wasn’t a smooth playback. This is due to the fact that my computer, at that instant, might be running slightly slower or faster than what it was running at when recording. Combine this with issues such as garbage collection stutters and you’ll have a messy replay.
 

Transform: Although stated above that you should be able to use floats on the same system, I wasn’t finding this to be successful. Unity’s transforms store position, rotation, and scale as floats. Because of the articles I read, and the errors I was getting, I eventually decided to implement my own transform component that only used integers. During LateUpdate(), the component would set the game object’s transform to the value of the integer transform.
 

After implementing this I found that I still had deterministic issues unrelated to transforms. So, it’s my theory that you could actually have a deterministic system even using floats (if you’ve done this, please let me know!). But, I haven’t tested that. Additionally, by ensuring that all values I record are not floats, I avoid any possible cross platform issues with any hardware that doesn’t implement the same floating point standard. Something to consider if you want to do cross platform online play.
 

STEP 3: IMPLEMENTATION

So now that we’ve avoided the non deterministic issues of Unity, how do we actually go about recording and playing back all this data?

First, we are going to implement a save state replay system as a debug feature. Although I’ve already decided I want the game to be deterministic, it’s very hard to test such a thing just by watching the game play through. I certainly don’t have the ability to determine if a ball bounces off of a platform on one frame or the next with my own eyes; and such an issue would cause a butterfly effect that I might not notice till minutes later in the replay.

By implementing a save state system, we can compare the simulation with our saved data and see if anything is off on the exact frame that we get out of sync. Not only will this prove our game is deterministic, but it’s also a great way to debug your game. Have an issue that’s hard to reproduce? No worries, you have the data saved to reproduce it exactly the same way every time.
 

To start off, we need to organize our game loop to be deterministic. I used the Update() function for this. In the game scene I have a GamePlayManager() class. This is what manages the update loop for all my game’s recorded objects. Since I need the game to run deterministically, I need to ensure that every recorded object's Update() function runs in the same order every time. As described above, this is where I’ll be using the UpdateAt() function, to ensure that each object is updated the same way, whether it’s recording or playing back.

Currently our Update() function in our GamePlayManager() looks like this:
 

void Update(){

       GetPlayerInput();

       CheckCollisions();

       object1.UpdateAt(); // object1 will not have an Update() function. Instead, we call this function when we want to update it.

   }
 

This will actually run deterministically (provided you solved the Unity issues above). But we need to ensure that this is the case. For recording, I’ve implemented a class called the VCR. The VCR has a few functions:
 

Recording: This will record player input. This data will be stored in a byte stream.

Playback: This will take a previously saved byte stream and feed input values, replacing player input. This will also set the state of objects while in debug mode.

Check Data: While replaying in debug mode, after we feed the player inputs and run a single frame of simulation, but before we hard set the state of those objects, we will compare what the simulation state is with what we recorded. This will let us know if our simulation is deterministic or not.
 

To perform these functions, every object you want to record will have to implement three functions:

SaveState(MemoryStream stream): This will record all necessary information about the object to the MemoryStream by converting it down to a byte array and writing it to the stream. For your input classes, you’ll record the value of each button the player can use. For object classes you’ll record values that change every frame, such as an object’s velocity, its state, and anything that is important for more than one frame.

RestoreState(MemoryStream stream): This will take a series of bytes and convert it to the value of the object’s variables saved above. For each variable, you’ll need some way to know how many bytes to read. In my implementation, I record everything in the exact same order, so I always know which variable type (and thus how many bytes) I need to properly set each variable.

AssertState(MemoryStream stream): This function will compare the value of the object’s current variables with what was restored in the previous function. How this works is we restore the player input, let the UpdateAt() functions of all objects run (so the simulation updates), and then call this function. This function will save the current values of all variables that have been updated from the simulation, then call RestoreState() to set the values to what they were when recording, and finally compare the simulation’s values to the recorded values. If the values don’t match, we know that the replay is out of sync. This will usually be caused by one of two issues.
 

Let’s add some functions to our update loop:
 

void Update(){

       GetPlayerInput();

       VCR.PreUpdate(deltaTime);

       CheckCollisions();

       object1.UpdateAt();

       VCR.UpateAt(isDebug);

   }
 

VCR.PreUpdate(int deltaTime): This function is where we set our random seed, save (or restore) player input, and set our frame’s deltaTime. I used an enumerated state value in the VCR to manage playback and recording, which would be set when starting the recording or the playback. When recording, call the input’s SaveState() function. During playback, call the RestoreState() function. During debug, call AssertState() on the input objects to make sure you’re playing values back in the correct order.
 

VCR.UpdateAt(bool isDebug): This is where we check to see if the simulation ran deterministically. So, if you’re not debugging, you don’t even need this function. When debugging, you will call the SaveState() function of each object affected by the simulation while recording. During playback, you will call AssertState().

Finally, when deciding to playback your game, it’s essential that the game is in the exact same state it was when you started recording. For this, I used the debug functionality to record the state of all objects at the beginning of our MemoryStream before actually running the game’s update loop. That way, I can set the MemoryStream’s position to 0, call the RestoreState() function of all objects before switching the VCR to playback mode, and can know that all objects are where they should be before playing back.
 

Now that you have a system that can save the state of every frame, you should be able to debug any determinism issues your game might have. Once you’re confident that the system is playing back correctly, turn off debug, and record and restore only the input values. If you’ve been diligent, then your game will play back exactly the same every time!

STEP 4: ADDITIONAL FEATURES
 

After getting the basic functionality down, Sportsball had reason to implement some extra stuff into the replay system.

Pausing: When pausing the game, I don’t want to record anything that happens on the pause screen. It wouldn’t be very entertaining to watch. Additionally, I want to make sure the players can also pause replays during playback. This got a little messy when I implemented it. You have to make sure that you don’t record the input commands that will pause the game. Additionally, you need to be able to run your update loop with the VCR in a new paused state. This generally means that you don’t call the UpdateAt() functions of any game object while the VCR is paused. Because of the order in which I record things, where I update values after accepting game input, I had to make sure to skip a frame of UpdateAt() after unpausing the game, since the input commands come before the VCR’s UpdateAt() function.

Chapters: Because Sportsball can get into a state where the player could essentially record forever (if the match goes into overtime and no one makes a score), I wanted to ensure that the game would never run out of memory. So, I implemented a chapter system, similar to DVD menus. Every 3 seconds I use the debug feature to record the state of all objects in the simulation. Then, I set a maximum number of chapters we can record. Once we reach that threshold, I shift every chapter back one, which will result in us overwriting the beginning of the replay. Although we lose the beginning of the replay, I can guarantee that the size of a replay will always stay within a set limit, even if the replay is recorded for years.
 

STEP 5: IMPROVEMENTS

Having written a replay system, I now have many features I’d love to implement, and a whole lot of refactoring. Although I won’t go into all the details, these are things you may want to consider when writing your replay system.
 

Order Manager: With the implementation above, saving dating into a MemoryStream, I don’t have any data in the stream about what was or was not saved. Because of this, I’m forced to save and load in the exact same order. While that’s not so bad, the issue comes when I add things to the game. If I add a new variable for a new powerup in the game that needs to be recorded the old replays will no longer work. As soon as that new value is reached to be restored, it will load the wrong information, and get the memory stream out of sync.
 

To fix this, I would likely create a sort of header at the start of the stream, where I can read a certain amount of bytes that will inform me what values I need to load from the byte stream and what those values are. Although this will increase the size of replays, it would allow me to use the same replay system to play replays with all sorts of configurations, and add new content to the game without breaking old replays.

Optimization: When saving data currently, I just always save the same data. For example, one value I record is the state of the analog stick. That’s 4 bytes of data each frame per axis. Even if the player doesn’t touch the stick for a minute, I still record a minute of 4 byte values. Instead I could add a boolean check to see if the value has changed. If not, I don’t record it and I could potentially reduce the size of the replay file. But, for values that are going to change every frame, this wouldn’t be necessary.

Component Based: Currently I pass in a list of objects into the VCR to be saved, which all have the SaveState(), RestoreState(), and AssertState() functions. This becomes rather cumbersome to add a new object to be recorded. Instead, I’d like to write a MonoBehaviour component that I can add to any game object in my scene. The VCR will then find all these objects and record their data. This way, my code is more Unity-like, and wouldn’t bloat up my classes with recording code. The system could be more easily dropped or removed from any aspect to any of my game projects. If I do get the time to implement this, then I can release the code as a separate package that any game could use, thus removing the need for this tutorial!
 

If you want to see the replay system in action, check out Sportsball, available on the Wii U November 6th, 2014. If you have any other questions about the details of this system, or if you’ve found any of my information to be incorrect, please let me know and I’ll update this article!
 

@piidx

auston [at] toodx.com