| HN Mirror

Of course. The reason expect-testing is good is that you need significantly less vigilance writing/maintaining the tests than when you do them with assertions for everything you care about, in exchange for slightly more vigilance required on the actual output of your programs. Yes you need to pay attention to the output, but your attention can now more focused instead of split between that job and the job of writing the test. It's possible to make mistakes when writing out your assertions, they are just generally more invisible and pernicious. Testing code is code like any other, and mistakes look like forgetting to test things, erroneous refactoring of the test or the code, mistakes copying tests around, mistakes writing out extrapolations, mistakes from sheer fatigue at the heft of the testing code you're trying to maintain. Further, the kind of vigilance required for expect-test is mostly not "Tesla kinda driving itself but driver is meant to watch the road". You are not checked out completely and talking to the other passengers or reading a book, but somehow legally responsible for taking control at any moment. You have your hands on the wheel and the car is offering turn-by-turn GPS directions.

Expect-testing is a good tradeoff in the short term (time to create tests) and in the long term (quality and size of test suites produced). The evidence for that is that there are pieces of software that need so many tests for their range of functionality, that you cannot test them any other way than in this style. I am talking about testing orders of magnitude more stuff than you could do manually. A great example is the Rust compiler UI test suite (https://github.com/rust-lang/rust/tree/master/tests/ui). It doesn't have to be that your tests have large amounts of noise, like compiler UI tests do. You can make focused and noise-free tests using this method, as the original post examined. The main thing is that writing the tests faster results in bigger test suites and more opportunity to look at the same code on different inputs. I would rather have two dozen tests that required me to look at their output, than three tests that made me think thoroughly about every single assertion. It's just a better use of your time. The rewards are compounded by the massively reduced cost of maintaining the test suite. The tests update themselves when the code does.

Overall, yes you have identified the negative part of the tradeoff. But you seem to have missed every single one of the benefits.