Bug negative verification

From www.pompidev.net

Bugs and debugging are a big issue for a programmer. Mostly because he\she should avoid them as much as possible.
I would like to share my thoughts about what I decided to call "Bug negative verification".

When fixing a bug, there are three important questions(or more) that a programmer should ask himself:
1) Did I really fix the bug?
2) Do I know why the bug happened?
3) How can I prevent similar bugs to appear in the future?
I want to talk mostly about the first question.

After finishing working on the last level of Labyrinthica: The quest of lima, I have been play testing to find problems. One major problem was that in the boss level, sometimes the walls were not drawn.
However, the physics worked and the mini map worked as if the walls were there. This led me to suspect that the class that made draw calls for the walls' sprites was part of the problematic code.
The code who made the sprite draw calls was a bit difficult to figure out how it will behave. I think that after profiling I tried to make it more optimized, but I am not sure.
So I wrote a simple and naive(KISS: Keep It Simple Stupid ;) ) code that suppose to have the same functionality.
At this point I wasn't really sure if the bug or part of the bug was in the code I rewrote. I play tested and saw the bug did not appear, but since the bug sometime appear and sometime do not, I wasn't really sure.

Then it occurred to me that if this code was not the problematic code, the old code should not performed any less than the new code. To check this, I have made both the old and new code run together with the same data(conditions) parallel to each other, and compare their performance.
The result of this check was that the new code was sometimes out performing the old code, but was never under performing the old code.
This is sort of the opposite of verification, <strong>I tried to verify that the old code was buggy</strong>.
Now I know there is a difference between the results of the new code and the old code, and that the points where there are differences, the new code performed better.
The conclusion is that the old code must be buggy. The new code might be buggy as well, but for now it can enjoy the benefit of the doubt that it outperformed the old code.

The next screenshot captures the bug symptom.
Billboard is the number of trees in the new code, Bug is the number of trees in the old code and Max is an upper bound of the number of trees in the old code.
You can see on the top right the mini map still works even though the trees are not drawn in the game itself.

BugVerification

 We sort of answered the question of "Did I really fix the bug?"
Trying to answer this question can help us answer the second question. But answering these two questions consumes a lot of time in debugging.
Debugging is a kind of a waste of time. Because you spend a lot of time trying to make things work, and not adding anything new to the program.
Is there a solution to this time consuming problem?
There is a partial solution. If we could learn enough from the bug, we could use this knowledge to figure out how to prevent similar bugs in the future, by taking certain design decisions and solutions.
I hope to talk more about how to prevent bugs in the future.