[inventitech]

I generally agree with you.

"sometimes easy testing is not actually useful and useful testing is cost-prohibitive."

While I agree here, too, I think that the situation where testing is not useful happens very rarely in practice. Everything can break, and when you think it cannot be possibly be wrong now, a future regression might occur.

I think badly written tests are a different problem. I.e. tests that for example assert on the serialized string and hence take a lot of maintainability effort to adapt to changing production code. But then there is nothing wrong with the test per se, its just badly implemented.

[IgorPartola]

I can think of lots of cases where testing is not useful in practice. Simulating random filesystem corruption when writing a Node.js app is pretty much useless: you don't know which files will be corrupted, and testing would take a huge (billions of lifetimes) amount of time to simulate all possible combinations even for a small program. Yet, this can (and in my experience did) take down a production system.

My other examples I already listed, I think, are examples of this too: you can test all you want against a spec provided by your external API maintainer, but if they don't follow the spec, you have a problem. We don't have a good framework (and I don't think we can create one), for testing layout problems in HTML that are caused by bad JS or CSS.

Basically, you do hit diminishing returns quickly, and you do get a false sense of security by having lots of small unit tests that don't actually prevent realistic failures.