Hacker News new | ask | show | jobs
by jagenabler2 12 days ago
How do you know your testing is complete if you're not sure what the underlying code does?
1 comments

Testing will never be complete.

  def plus(a, b):
    return a + b
The above code, if a and b are integers have 2^64 test cases for full coverage. Code being much more complex then that on average will have a magnitude more test cases needed for full coverage... likely more test cases then all of the code humanity has ever wrote since the dawn of programming.

Testing is a statistical sample. You sample a small subset out of the full domain of possible tests and that sample you hope is somewhat representative of the actual population of tests. To put it in a sentence: If my suite of tests is correct then it is more likely that the entire full universe of every single possible test for my program can all be correct.

Realistically though the universe of all possible tests is so large that it's basically impossible for any sample to be a good representation of it. A suite of test does help up confidence that things will be MORE correct, but the sample size is so small it is almost guaranteed that you missed something.