I have an application that is returning the wrong output, when it is run with a particular input choice. I haven’t been able to get anywhere near a diagnosis of the fault, despite spending about a day and a half on it. If it is run with Input A then it gets the correct output, if it has Input B then the output should be the same but is incorrect. Input B goes through some slightly different processes to Input A, in an attempt to optimise it, although most of the steps are in common.
I am trying to compare side-by-side an invocation of the program with the two inputs. Its a difficult way to debug as you have two application windows, two visual studio windows, and its easy to get mixed up.
The software is about 1 million lines of code, and some of the code is fiendishly complicated. It has many nested loops and loops that run for a lot of iterations, with very complex variables. It caches a lot of data, so when you find that values in one mode are different from the other you have to work out if the differences are significant, and then its hard to trap where the programs diverge since the values have already been calculated and you have to keep restarting the programs and going back.
I understand there is no silver bullet here but I just wondered if anyone had any tips?
5
The answer is simple, the execution is complex.
Basically what you’re trying to do is identify the point at which the variance occurs i.e. the point at which the actual result starts to move away from the expected result (and if this sounds like something that might show up through coded tests (“unit” and up) then that’s not entirely surprising). Similarly you need to identify the point at which the variance stops i.e. where do the paths join such that you’ve got a result that changes consistently again. Having identified those points you can work to further narrow down the possible locations for the “errors”.
The brute force way is adding debug/trace statements, breakpoints and inspection and then running the code.
You improve on this in two ways:
- First by reducing the amount of code you have to run – there are diverse means. (As suggested, this means isolating/separating out the problematic code.)
- Second by wrapping the stuff in tests (which is probably the best of the diverse means
if feasible).
Sadly none of this is straightforward or necessarily quick.
3
Its a difficult way to debug as you have two application windows, two visual studio windows, and its easy to get mixed up.
Really? You have two monitors. Put one variant on one screen and the other on the other. If you don’t have two monitors, then welcome to the 21st century. They’re like $100.
I understand there is no silver bullet here but I just wondered if anyone had any tips?
So it sounds as though you have a overly complex (read: poorly designed/implemented) codebase. And it sounds as though you have no unit tests.
Ideally, I would look to simplify the codebase. Pass in dependencies to loosen coupling. Break big functions into smaller ones. Cut out side effects. There are piles of good practices to do that, but since that is “how to write good code” – I’m not going to go into it in depth. And really, you shouldn’t simplify the codebase until you get the bug fixed.
Unit tests though will help. They can isolate the bug, and help you debug just that isolated part rather than the whole complex app. They provide clear input/expected-output pairs. They prevent later users from breaking things. And they will put pressure on you to write good re-usable code. Granted, it might be infeasible to make good unit tests if your code is overly complex.
2
I have not much experience with Visual Studio, but in many IDEs you can keep a Watcher on a variable, and step through the code while keeping an eye on its value. Here is the MSDN documentation on a feature that seems to be equivalent to what I remember from other IDEs.
Input B goes through some slightly different processes to Input A, in an attempt to optimise it, although most of the steps are in common
It seems to me you should put a big, fat breakpoint at step 1 of the first ‘slightly different’ processes, and then step from that point on when debugging input B. (Compare its value to the value of input A at the previous step, i.e. before those ‘slightly different’ processes started, so as to be sure that the processing up to that point was the same.)
Another, quite subjective advice: if stepping through so many lines of code is really time-consuming, you may use a ‘binary-search’ approach once you know which set of code lines are involved (probably from line 1 of the ‘slightly different’ processing to the last line). Put a breakpoint in the middle of that interval; check if A and B are different; if they are, put a breakpoint in the middle of the interval before that line; if they are not, put a breakpoint in the middle of the interval after that line. If you do not know which lines of code are involved, then stepping ahead from the first point where processing is different is your only option.
3