Unit tests vs. functional tests – round 101

Picture this – lazy Saturday afternoon. You’re just watching your favorite Hannah Montana episode, when suddenly a beam of bright light hits the eyes of your intellect.

Squinting, you force your sight into the light and you suddenly find yourself “in the knowing”. Everything is so clear, the gears of your mind start spinning like crazy.

You jump right off the couch, kicking your PC online on your way for a large pot of coffee, then you jump into your seat and start coding. You’re in the zone, the place oh so comfortable – and lines of pure genius start filling your screen, piling up.

And then comes the moment – you press play, the app starts just to show objects flying about in the chaos of errors.

You feel bad, you start debugging things – I mean you had it right there, it was so clear and simple, how could you possibly make a mistake. But The Evil God of Code is not forgiving – he’s only started testing you. And you’ll soon discover that there’s a lot more to it than the basics you wrote.

You took the bait, and now you’re being slowly reeled in, as you fight to prove your concept working.

The fall

I didn’t make up that story. I recently wanted to add a very simple feature to my engine – a skeleton mapper.

“It’s such a trivial thing” I thought “and I’m gonna keep it as simple as possible – any editors or functionality extensions will come later“.

In my engine I already have a well established animation system and a blend tree implementation built around it.

So I started simple – by blocking a set of unit tests that would establish the initial interface and help me get the mapping math right.

Long story short – I made the initial set of tests pass, but when I ran the code in Tamy (my game engine editor) on the sample assets I had, it broke spectacularly.

Not gonna bother you with details, which I will describe at some point in a post dedicated to the skeleton mapper’s design, but as it turned out the problem lay in the configuration of the target skeleton, and the fact that my actual assets were far more complex then the simple configurations I set up in the unit tests.

The struggle

I started thrashing about, spitting out more and more tests, and as soon as I had them all in green, testing the mapper in the editor, just to learn that there were other problems I needed to solve.

Soon, I found myself tired and forcing to do more code and more tests.

What started as a fresh breeze of inspiration quickly turned into a labor camp.

And at that point it struck me! I took a pause, stepped away from the keyboard, and when I returned after a while, I learned that:

“How this could have happened?” was my first though. 2 options of breaking out of the vicious circle immediately presented themselves.

The second one seemed more appealing, because it had the added benefit of me ending up with an editor which I would have to build at some point either way.

And the winner is…

I’m gonna stop reminiscing at this point.

Like I promised, I’ll tell you all about my SkeletonMapper design in another post.

What you need to know is that after building the editor and testing the section, I finally found a satisfactory generic solution to all configurations I managed to come up with – and they did live happily ever after…

What I want to focus on right now though is finding a answer to the following question – how could I have done it better?

To sum up:

So knowing all that paints a clear picture of how I should go about things.

But WHY did it work out like that? I mean unit tests are a well tested method and the test cases I had drawn weren’t bad.

The problem there was – I tend to keep my unit tests simple and illustrative, which translates directly onto the data configurations I use for testing.

And that was the reason I hadn’t been able to test a larger system with data that might be arbitrarily different ( let’s face it – when I’m building a skeleton rig in Blender, I don’t think about the order of bones etc – I’m focusing on its application to animation, a different point of view altogether).

The other problem was relying on the test to be able to tell you the best way to build a mapper, when later on you’re going to change that either way by introducing a dedicated editor.

One might of course argue here that if the tests had been done well, they would have led to a design that could be reused as an API to the editor.

And rightly so – I even had it running like that at first.

The problem was that it was SO inefficient.

The primary objective of the API I developed in the test harness was to be able to quickly and in as visual manner as possible configure a mapping between two skeletons.

Meanwhile, in the editor, you want to do all sorts of other things and get access to different aspects of the mapper etc. which in this particular case effectively rendered the original API useless.

The world keeps on turning

The lesson learned here was to pay attention to what I’m supposed to unit test.

Unit tests are a major asset when it comes to math, message flow and testing the insides of a system.

Establishing the data interface though – that’s best left to an editor.

First of all – you immediately get access to it during functional tests and you can start playing with your system really quick, building new sets of functional tests, and come up with new cases to solve with the unit tests.

And it will keep you busy and interested, switching between doing different things as opposed to being stuck looking at the debugger all the time – gives you more options, and that’s always good!

I hope this turns out helpful to you as it was for me.

And stay tuned for more stories from the trenches.

Because war – war never changes ;)

--------------------------------------

This post was reblogged from https://ptrochim.wordpress.com/2015/01/12/unit-tests-vs-functional-tests-round-101/