Directing Virtual Characters in Wayward Strand, Part 1

After my last post, on how we’ve integrated ink and Timeline, there was some interest in how we use Unity's Timeline tool to do things like trigger animations on characters.

Over the course of making Wayward Strand, we've developed a solution for this that we're pretty happy with. Put simply, we use Timeline to set parameters on our characters' Animator Controllers  - but we've developed quite a bit of scaffolding around this, as well as a useful categorisation structure for our animations.

There's quite a bit that goes into all this, so I'll cover it over a couple of posts  -  this first one will focus on the Animator Controllers, and I'll start by going into the reasoning behind why we went for this approach.

Our problem space for animations in Wayward Strand is borne out of our game’s unique, simultaneous story structure. We wanted Wayward Strand to feel like an interactive theatre experience like Sleep No More, where the continuous location and physicality of the actors is such an important part of what makes that experience unique.

You see the character in their highlighted, plot-critical moment, where they perform the key action, or have that impactful interaction, that their story is structured around - but you also see the character after this moment, when they have to move, in character, from one location to another. You also see them in their quiet moments, where they spend time in a room or space feeling satisfied, confused, expectant, or disconsolate - sometimes for considerable stretches of time.

In a real-time, criss-crossing story structure, these wobbly, windy arcs of heightened action and quietude/reflection are a joy to experience, and part of that joy comes from the choice you make in who to follow - who to spend time with at any point in the story.

In a linear storytelling format, like a film, whenever you have a quiet moment, it’s because the director decided that you’ll have a quiet moment at that specific point in the story. Your moment-to-moment experience has been foreseen; planned; designed. But in a simultaneous storytelling format, you’re choosing to have that quiet moment, or happening upon it, while still existing inside the story’s world.

In attempting to capture this feeling in a videogame, we knew that we wanted our characters to live as continuously as we could manage within the world. This means that they’re generally not snapping from one state to another - the camera isn’t cutting away from them and then cutting back, allowing you to snap their animation state, or even load in an entirely new version of the character to handle a particular story moment.

Our characters exist in the world - performing a variety of actions, participating in scenes, having their quiet moments - and they do so continuously throughout each day, regardless of whether the player is near them or not.

The solution we landed on for Wayward Strand was a structure where we split most animations into belonging to one of two categories - BodyPose and Activity. Using this logical structure, we realised that we’d be able to run a lot of complex character movement through systems, without having to worry exactly about how the movement is managed on the code-side. This categorisation also helped significantly in determining which animations to develop, and what their requirements were.

I want to give a shoutout to Kalonica in particular here, who really came up with this logical solution as we were discussing our requirements and desires for the game, and convinced me of certain elements which I had thought would be limiting, but haven’t been in practice, and have actually turned out to be a really important part of having a feasible system come together that actually works.

The BodyPose/Activity solution required a very specific structure inside Unity’s animator controllers, and a setup where there is one Parameter directly linked to each BodyPose/Activity, named exactly the same as that BodyPose/Activity. There’s a bit to go over, so I’ll describe the way we set up the AnimatorControllers, then we’ll go through the parameters and how we trigger transitions between different states, and finally we’ll see the whole system in action. We’ll be looking at the animations of Ida, one of our characters, and by the end, we’ll be breaking down this set of transitions, where Ida goes from standing, to holding her knitting up to inspect it.

A BodyPose is generally a fixed state in which a character could do a lot of different things - Standing, Sitting, Lying in Bed, etc. These BodyPoses live at the top level of the hierarchy, and each is represented by a StateMachine.

A general rule that we follow for BodyPoses is that they go through their default BodyPose when transitioning from one to the other (the default is Standing for ambulant characters). I think we’d be able to break this rule if we needed to, but so far, it’s made sense - because if a character is transitioning from one BodyPose to another, generally they’re moving from interacting with one prop, to interacting with a different one.

Let’s have a look inside the Body_SittingLowChair StateMachine.

Each BodyPose (except for Standing) is generally structured in this way - there’s an In State/Animation, a Doing State/Animation, and an Out State/Animation. The In plays its animation almost to the end, then transitions to Doing, and Doing plays a looping animation, and will transition to Out when the Body_SittingLowChair parameter is no longer set.

From the In and Doing states, there are transitions also set up that go to each of the Activities that are in the BodyPose - and from the Activities there are transitions back to Doing, or directly to Out, to allow for smoother transitions when going from a BodyPose+Activity to an entirely different BodyPose.

I can see most people looking at this (especially if you’ve worked with Unity’s AnimatorControllers before), and those overlapping transition arrows, and thinking “oh god, that looks like a nightmare”. And.. it kinda is! Which is why, after we had defined this structure and decided to invest in it for the majority of animations across all characters, we developed some EditorScript tools to help us automate the process.

In the following gif, I’m using one of the buttons we’ve added through EditorScripts to set up the default transitions for the Activity - the transitions to and from, including those additional ones from In and to Out. (The script also adds any relevant parameters to transitions where necessary.)

Even though it takes some time to figure out and implement Unity’s editor scripting, automating these kinds of processes is really handy, because it ultimately saves time, reduces the chance for errors that you have to track down, and gives you some good defaults to work with - each of those states/transitions can still be modified after the script has gone through and done that initial set up, but you’ve got a baseline that will work in game.

Okay, let’s go one layer deeper, into the Activities themselves. One of our most complicated examples is Ida’s animations for doing her knitting.

You can see that we start out with a similar structure to the BodyPoses - In -> Doing -> Out - but that instead of a deeper level of specificity in here, we have a bunch of States/StateMachines that are linked to generic parameters - Pause, SpecialLoop, SpecialOnce, etc.

For the additional States at this level beyond Doing, we’ve landed on the term ActivityModifier - these animations “modify” the Activity, but the character is still fundamentally, logically within the Activity (and the associated BodyPose, one level up). ActivityModifier is a bit of a fiddly name, but works for our purposes. We start out all of our Activities with this basic structure, which allows us to add specific animations that are driven by generic parameters.

For Ida’s knitting animations, she can start knitting (In), loop knitting (Doing), and stop knitting (Out). She can also rest from her knitting loop without exiting the Activity (Pause). This Activity also has three special states - Ida lifts the knitting up to inspect it (SpecialLoop), Ida puts the knitting down into her lap (SpecialLoop2), and a SpecialOnce State that isn’t currently used, but could be used for a ‘once-off’ animation, like one where Ida sighs while knitting, or realises she’s made a mistake and back-tracks in her knitting a bit.

The reason these states are named in this generic way, and use generic parameters, is so that we can trigger the parameter regardless of what BodyPose/Activity the character is in. For example, the modifier we use most often is the Pause modifier - it’s great to have these Activities set up such that a character can doing an Activity (like knitting, or reading, or eating), and then pause the doing of that activity so that they can eg. respond to someone, or can just pause to take a moment within a scene.

In Wayward Strand, there are scenes that are triggered by the player’s presence (by Casey asking a character a question, or a character talking with Casey because she’s in the room), where we don’t know exactly when the scene might play out. But, by using generic parameters like Pause, we can still direct Ida within the scene, during a conversation, to Pause or resume Doing, regardless of whether at that time she happens to be knitting, or drinking tea, or doing something else.

This means we can create nuanced portrayals of characters, even when we don’t know what they’re currently doing, through using these generic ActivityModifiers, along with other elements of character direction, like specifying which way their head is facing (whether Ida is looking at Casey or whether she’s looking at whatever she’s doing), and other animations which might only affect characters’ heads, like talking, sighing or nodding.

Returning to the gif from the start of the post, we can see this in practice. When Casey asks Ida about the photograph in her room, she first pauses her knitting, then puts it down into her lap as she responds.

In the next post in this two-part series, we’ll explore the parameters that drive the transitions inside the Animator Controllers, as well as how all of this integrates with Unity’s Timeline tool.