Hacker News new | ask | show | jobs
by YourDadVPN 889 days ago
What is your opinion of code coverage requirements now? I have been in a "phase" of seeing them as "code quality theatre". Considering that a function which takes a single 8-bit integer as an argument already has 256 unique inputs, and may bug only on 1-2 values, 100% statement coverage can be very misleading. A typical function has billions or trillions of unique inputs and 100% statement coverage could be very nearly 0% state space coverage. I'm 5y into my career (but 15y into programming) and aware that my opinions will change and develop as I progress. This one has been stable for a while though.
1 comments

Coverage is necessary but not sufficient, your tests also need to be good and test the right things. What's your proposed alternative - that we don't even try because our tests aren't guaranteed to be perfect?
Well, almost. I use a lot of assertions to check the obvious (value out of range, null pointer, etc.) and test the happy path(s) to prove the code at least works in the cases I can anticipate. I add unit tests for complex algorithms, and to prove a reported bug is what I think it is and that it has been fixed. Otherwise, I think using unit tests to find bugs is mostly busywork.

For even a fairly trivial piece of code, the search space for bugs can be vast or even infinite. Writing unit tests to find bugs within that space is like throwing darts at an infinitely large wall and trying to hit an unknown number of invisible targets. You can only write tests for the potential bugs you anticipate - if you could anticipate a bug, you wouldn't write it, right? You end up with dozens or hundreds of tests that probably never failed, except when you have to change something. Such was my experience when I tried to maintain high code coverage. When I switched to writing assertions and acceptance tests, my rate of bug reports did not change, and I was more agile.