Exorcising Satan's Rotoscope: Motion Capture from an Animator's Perspective

Author: by David Stripinis

Motion capture. Perhaps no single technology frightens animators more. Whether it's the technical hurdles or the perception that it will put them out of a job, many animators have a fear and loathing of motion capture. The truth is animators have nothing to fear from mocap, except avoiding it. That will definitely lead to putting you out of a job. My prediction is that within the coming generation of video games, nearly every title will involve at least a modicum of captured motion. Animators will need to learn to, if not embrace motion capture, to co-exist with it.

We are entering into a new generation of gaming. Rather than dealing with a blocky low-polygon guy jerking by at 12 frames a second, we have highly detailed character running around at a blazing 60 frames a second. Simple cycles baked out at 15 frames a second won't really do the job anymore. Animators must find a way to fulfill the desire of gamers to see motion as realistic as the character models being animated while satisfying the budget requirements of the project.

What Motion Capture Can Do

Motion capture does one thing and does it extremely well -- and that is take the movements of a performer and put them in the computer. That's it. You can capture pretty much anything you are able to stick the markers to. Whether that's Kobe Bryant, a horse or even you doesn't really matter to the computer. It's just a bunch of dots moving through space.

So why is it any good at all? To be slightly redundant, it does reproduce that motion quite well. Whether it's the distinctive footwork of a NBA all-star or the subtle shifts in weight and movement of woman standing around seemingly doing nothing, realistic motion can be deceptively complex. An animator can convince you something is alive. A truly skilled animator can even reproduce many of the subtleties of real movement.

But at what cost? Why have your best animator (or in many smaller studios cases, only animator) spend a week animating all the subtleties of a motion that would take 20 seconds to record on a stage? That is a tremendous waste of time, talent, and money. It also isn't much fun to the animator. Most animators would rather spend their time animating the death of that nine tentacle slime beast than another $&*#! walk cycle.

What Motion Capture Can Not Do

This is not to say that motion capture is ideal for every animation need, even for realistic animation. If your title calls for cartoony animation, motion capture is not your best option. And even if you need realistic motion, there is a good chance your game will need some elements that cannot be captured. Be it swimming, suffering a serious injury, or an imaginary creature.

Motion capture is not a plug and play technique. You will need to workaround it's shortcomings to effectively put the technology to use.

Capturing the Motion

Know what you don't know.
This is by far the best advice I can give on getting good motion capture from your vendor. There are a lot of technical issues with motion capture. Don't try to help with them. Let the people who do this day in day out take care of it. Ask questions, but defer to them on technical issues. It's hard, but it'll lead to better captures. Make sure you get the end result you're after, but the in between issues are best left to them.

Be prepared. (Apologies to the Boy Scouts)
Know exactly what you need, prop wise, set wise, etc. That way they can have everything on hand and ready.

Realize you are, in fact, dealing with a real live human performer.
Hopefully you were on hand for the talent auditions. This will definitely help on the first day of the shoot, as you will at least have some knowledge of each other's personalities. Your performer is neither a performing monkey nor robot. They will get tired, they will get annoyed and frustrated at doing the same thing over and over and over, and they may not understand exactly what you want. Keep this in mind as you work with them. After your first day, realize how tired you are at 6:00, and then realize you just sat around. They jumped through hoops all day (maybe literally). Make your shooting schedule conform to the realities of working with a person. Don't put your most physically demanding shots first or last. Give your guy time to warm up, but not get too tired. Keep their health in mind. If you see them rubbing their shoulder or ankle, give them a 15-minute break. Keep in mind these guys are (usually) trained professionals. They know how to hit a mark and do the same action with remarkable precision over and over again. Use their knowledge and expertise to your advantage. You may not know how to use a missile launcher, bastard sword or do a triple back handspring. They might.

Stay in control.
You are spending a good bit of money on this. You should expect to get what you want. While you should always follow the Golden Rule and #3, make sure you leave happy and with everything you need. Make sure you check the actual capture data on the system. Most modern systems provide near instantaneous replay of the motion. Even if the performance was perfect, a loose marker can mess up the final data.

Scheduling is everything.
Schedule an extra half-day of stage time to get everything you need, anything you forgot (you can USUALLY add extra shots) or give some breathing room for any delays. After all, if that time goes unused, that's still cheaper than if you have to come back for re-shoots later.

That's the basics of a good mocap shoot. A lot of it is obvious, but it's usually obvious AFTER the fact.

Mapping Motion Capture

Now that you have your motion data recorded, the real work begins. Good motion capture is made or broken once you get the data.

Motion capture, until recently, required you to build your skeleton and character around your captured performer. Thanks to re-scaling and mapping tools such as House of Move's Dominatrix, artists now have the capability to adjust the proportions of a captured performer to that of a pre-defined skeleton. Obviously, picking a performer that is close to your characters proportions will help add credibility to your motion.

Layered node systems such as Dominatrix also allow for the ability to edit motion capture in a non-destructive fashion but the additional transform nodes, as well as the data density of the original mocap makes using this hierarchy as your final in-game skeleton difficult. Also, the joint orientations of your previously constructed character's skeleton may not align with the capture skeleton. For instance, the bending of the elbow may be the +X channel on the capture skeleton, and -Z on the character's skeleton. On top of that, due to the realities of Euler math, what appears to be one axis of motion may be two or all three.

Using some basic math and scripting, you can determine the difference between the axis's of each joint, both the XYZ orientation and any angular variations. You can then, frame by frame, get the rotational values of the captured data, apply the difference to it, and write that to the target character's skeleton.

The Animation Workflow

On top of all the concerns an artist has getting characters to move the way you want, game animation has special concerns. Among these, memory footprint and cyclic motion are perhaps the two most significant. Unfortunately, the data provided by motion capture is anathema to both of these qualities.

Motion capture provides a keyframe for every joint, at every frame. This data density hampers editing of motion in any form. If at any frame you need to alter the motion, that keyframe is sandwiched between other keys. This creates a spike in the motion curve, which leads to a twitch in the motion.

Deleting the keyframes on either side of the frame we need to alter, smoothes out the spike, but still provides a noticeable twitch. While these alterations and twitches are problematic in the middle of an animation curve, the can be devastating to a cyclic motion. In order to provide a cleanly looping motion, the start and end pose have to align, as well as the in and out tangency of those poses. While we can continue to delete and tweak keys, this can be a laborious, time consuming process. While some off-the-shelf software provide tolerance based curve simplification, this often provides even more convoluted results, removes all the subtleties motion capture provides, or simply doesn't work.

The other major problem presented by this data density is the memory footprint of animations. For example, a simple skeleton with only 24 joints would require 4320 rotational keys per second at a frame rate of 60. Extrapolate that across the amount of animation a character requires, and suddenly a whole lot of data needs to be held in memory. Dedicating 40% of your available RAM for one characters motion wouldn't be acceptable in any game, even a character centric genre like fighting.

Animations created in the traditional method obviously do not suffer from this data density. Keyframes are added only where they are needed. Skilled animators can tweak and adjust the in and out tangency of the keys to optimum effect.

The question then arises, why do we insist on chiseling away at motion capture data until it's usable, when we could just add what we need?

Additive Curve Fixing

To this end, I developed a process I call Additive Curve Fitting. Once motion capture is mapped to a skeleton, it is fed into a "data holding tank". This data tank can be anything that can hold keyframe data, what that holder is can vary from software to software. For instance, in Alias|Wavefront's Maya, the software which served as the development platform for this technique, adding dynamic attributes to each joint in the skeleton containing animation data.

From here, the animator has complete control over where and when keys are added. A simple press of a button queries the data held in the data tank, and sets a corresponding key in the "real" channel. This data includes not only the obvious time and value of the key, but the tangency as well. Of course, your in game animation system will have to support these features to take full advantage of this.

To use this technique, an animator can slide back and forth in time, adding keys from the data tank, or by hand posing the skeleton. This allows for infinite control and editing of any aspect of the motion, while being in precise control of the data density of the final game asset. In the case of our in house export and in-game animation system, Bridge and Olema, we can even retain all the motion capture data in Maya binary file, without increasing the size of our export. Because of this, we can easily and quickly add keys to our "real" animation curves, balancing export size and motion quality.

If this methodology sounds familiar, it is effectively what any animator does. Breaking down motion into key poses, then layering on the details. The major difference here is that the pose is pulled from the motion capture, not the animator. Of course, at any time the animator can alter the pose to refine or edit the motion. And because we are working with motion capture, we can be assured the timing of the animation is true to life. This dramatically speeds up the asset creation. A first pass at an average asset takes a skilled animator 30 minutes to an hour, usually with an additional 2 hours of refinement to a final in game asset. Other difficulties associated with using motion capture for in game assets, such as looping, are no more challenging than with a traditional animation methodology.

As you can see, motion capture is nothing to fear. It allows for a speedy and cost effective solution to realistic human motion. It improves our skills as we deal with the accurate timing of recorded motion. But most importantly it frees us as animators to concentrate on animating the things which can't be captured. True, it presents some technical challenges, but they are not insurmountable. Motion capture is not the tool of evil many animators make it out to be. It is a tool of the digital age, and like any tool, once you learn how to use it effectively, you won't remember how you lived without it.

References

Understanding Motion Capture for Computer Animation and Video Games
Alberto Menache
Morgan Kaufmann Publishers
ISBN: 0124906303

The Illusion of Life: Disney Animation
Frank Thomas and Ollie Johnston
Hyperion
ISBN: 0786860707

Special Thanks to Tom Harper Alias|Wavefront, Bryan Ewert H20, Julian Mann Double Negative, Andre Bustanoby Digital Domain, the whole of House of Moves, and most especially Factor 5.