Can anyone comment on the claim "QuickCheck is shockingly more effective at finding bugs than unit tests"? I'd be interested in hearing other opinions.
It's essentially a unit-fuzzing framework, so it can find holes in boundary conditions people generally forget to check, because it's in their blind spot or because "nobody would ever do that".
QC does not care, and since it's injecting fuzzed data at the unit leve (~function, generally) it makes it quite easy to see precisely where the failure happens.
I haven't used QuickCheck, but read about it. So take this with a grain of salt, but it seems to me that specifying properties that should hold for your functions and then letting QuickCheck throw all kinds of data at them would be more efficient at least, if not more effective, than manually writing a bunch of unit tests and coming up with your own test data.
It's worth noting that QuickCheck is used with functional code only. For non-functional code with side effects (e.g. database transactions), Haskell has unit test frameworks that are not too different from unit testing frameworks in other languages.
What distingishes QuickCheck from regular unit testing is that you give it a function to test, some invariants that must hold and an input generator. Some of this is automated with compiler magic (where possible) to make simple tests really simple to write. QuickCheck will then grind through a bunch of random inputs and see if the invariants hold, finally showing you the failed cases.
It's surprising how often people overlook weird edge cases that QuickCheck finds, because it doesn't have as many preconceptions about what kinds of inputs are sensible.
The claim is true: QuickCheck is surprisingly more effective at finding problems (and documenting expected behavior) than unit tests. When I first tried it, not long after the QuickCheck paper came out at ICFP 2000, it found problems that I never would have thought to test for. [1] (That's why I end up writing something like QuickCheck when I code in environments that don't already have one. [2] It's worth it.)
The key to its effectiveness is that it doesn't suffer from limited human imagination. When I write test cases by hand, it's up to me to think of all the things that could go wrong and write test cases for them. But, being human, I have blind spots. Even when I try to be systematic about detecting edge cases and writing tests for them, I miss some. But when QuickCheck-like tools write the test cases for me, they can dream up corner cases like you wouldn't believe.
And, even better, when I use QuickCheck, my test code doesn't end up looking like an enumeration of corner cases. Instead, it becomes clear, concise, and formal documentation. (For a good example of QuickCheck properties as documentation, see [3].) I just specify the intended general properties of my code, and QuickCheck generates the messy test cases behind the scenes, where they don't become visible noise.
It tends to be best for mathematical code because you can quickly think of invariances.
In any situation where you can state a powerful invariance your code must always be consistent with it's both fast and thorough. The common example is that (= str (reverse (reverse sequences))) for all sequences.
Note that this only tests invertibility. There are lots of ways reversal can be broken without violating this invariance. It's just really easy to write down this invariance and use it to rapidly check gross violations.
QC does not care, and since it's injecting fuzzed data at the unit leve (~function, generally) it makes it quite easy to see precisely where the failure happens.