Beautiful Debugging > A Systematic Process

28.2. A Systematic Process

When programmers debug a program, they search for a failure cause, which could lie in the code, the input, or the environment. This cause must be found and eliminated. Once the cause is eliminated, the program works. (Should the program still fail once the cause is eliminated, we may need to revise our beliefs about the cause.)

The general process for finding causes is called the scientific method. Applied to program failures, it works as follows:

  1. Observe a program failure.

  2. Invent a hypothesis for the failure cause that is consistent with the observations.

  3. Use the hypothesis to make predictions.

  4. Put your predictions to the test by experiments and further observations:

    1. If experiment and observation satisfy the prediction, refine the hypothesis.

    2. If not, find an alternate hypothesis.

  5. Repeat steps 3 and 4 until the hypothesis can no longer be refined.

Eventually, the hypothesis thus becomes a theory. This means that you have a conceptual framework that explains (and predicts) some aspect of the universe. Your failing program may be a very small aspect of the universe, but still, the resulting theory should neatly predict where you should fix your program.

To obtain such a theory, programmers apply the scientific method as they walk back through the cause-effect-chain that led to the failure. They:

  1. Observe a failure ("The output is wrong").

  2. Come up with a hypothesis about the failure cause ("The problem may be that y has a wrong value").

  3. Make a prediction ("If y is wrong, its value may come from f() in line 632").

  4. Put their prediction to the test ("Indeed, y has a wrong value in line 632").

  5. Draw appropriate conclusions ("This means that f() returns a wrong value. Now where does this come from?").

Among all methods, hints, and tricks, the consistent and disciplined use of the scientific method is the key to becoming a debugging master. This means three things:


Be explicit

Formulate your hypothesis explicitly. Write it down or explain your problem to other people. Keep a written track of your hypotheses and observations so you can interrupt your work and start the next morning with a fresh mind.


Be systematic

Make sure you know what you're doing. Don't investigate (or change) something at random without having a clear hypothesis and a resulting prediction. Be sure you do not miss possible failure causes.


Look for the most likely causes first

The scientific method guarantees you will find a cause, but it does not tell you when. Identify possible failure causes first, and then focus on those where the product of likelihood and effort is minimal.

Unfortunately, interactive debuggers as they stand do not support the scientific method. To be sure, debuggers are great tools to poke around and investigate code and its results at will. This is a good thing—but only for skilled programmers who know how to use debuggers systematically. I'd rather see programmers trained in systematic debugging methods than in fancy debugging tools. (And I still feel guilty having written a fancy debugging tool myself.)