[bzbarsky]

The amount of test code and test data is proportional to the number of possible codepaths through your code.

This is generally exponential in number of functions, modules, etc involved. For example, a function with N if statements that are not nested generally needs 2^N testcases to properly exercise it.

So having 1000 times more tests than code may not mean that you have complete coverage at all. It depends on the structure of the tests and the code.

[scott_s]

I think your analysis is for executing every distinct code path. I read through the SQLite testing page, and they claim 100% branch coverage, which means that they test every possible outcome of a branch - but that's different from what you're going after, which is every possible code path.

(Not disagreeing, I just had to go through this process in my head when I thought about what you said in comparison to what they said.)

[bzbarsky]

Indeed. What they're doing is much better than what most software projects manage, but not quite enough to test correctness unless the code in later branches is completely independent from the code in earlier branches....

For any nontrivial project, testing every codepath is basically impossible, unfortunately. :(