[asgard1024]

> Tests that use random input data are much more difficult to write correctly.

Interestingly, I personally find them easier to write. I actually find classic unit tests hard to write, probably because I am painfully aware of the lack of coverage.

While with property-based testing, I start from the assumption I have on what the code should do. Then the test basically verifies this assumption on random inputs.

Doing unit test with the given input seems to me backwards - it's like a downgrade, because I always start from what kind of assumption I have and based on this I choose the input. And why not encode the assumption, when you already have it in your mind anyway?

[slavik81]

Your implementation is necessarily complex. That's why it may have bugs, and why it needs tests.

You have many more tests than implementations. In my experience, ~20x more. If your tests had bugs at the same rate as your implementation, you'd spend 95% of your time fixing test bugs and 5% fixing implementation bugs. That's why tests should be simple.

If you're going to be spending that much time on validating assumptions, I think you're better off trying to express them formally.

[asgard1024]

I think I disagree, but it really depends what you mean by "test" or "test case". I assume that test case is for a given input, expect certain output, and test verifies certain assumption, such as for a certain class of inputs you get a certain class of outputs.

I believe that you always test two implementations. For example, if I have a test case for a function sin(x), then I compare with the calculator implementation, from which I got the result. So if the tests are to be comprehensive (and automatically executed), then they have to be another implementation of the same program, you can't avoid it, and you can't avoid to (potentially) have bugs in it.

Now, the advantage is that the test implementation can be simpler (in certain cases); or can be less complete, which means less bugs, but also (in the latter case), less comprehensive testing.

In any case, you're validating the assumptions. The assumptions come from how the test implementation works (sometimes it is just in your head). And to express them formally, of course, that's the whole point.

For example, if you're given an implementation of sin(x) to test with, you can express formally the assumption that your function should give a similar result.

By formalizing this assumption, you can then let the computer create the individual test cases; it is a superior technique than to write test cases by hand.