[mempko]

The client doesn't run your unit tests, but will run you contracts (because contracts deploy with you software while unit tests dont).

You might have realized users of software do things the engineers don't expect, which are not covered in unit tests.

[dataflow]

> The client doesn't run your unit tests

That's kinda the whole point... you run them to catch bugs and fix them so your clients never see the bugs you caught in the first place.

> but will run you contracts (because contracts deploy with you software while unit tests dont).

So you'd prefer your contracts to blow up your bugs in your clients' faces, rather than catch bugs yourself prior to releasing the code to them?!

[mempko]

You would obviously test the software before it goes to the users. With contracts you would be able to have information from your users that make it MUCH easier to fix. Without contracts you will have a much harder time fixing the bugs. Contracts catch bugs unit tests don't.

[dataflow]

> You would obviously test the software before it goes to the users

That... is literally what unit tests do. Test the software before you give it to users.

You seem confused what the debate is over. Nobody is arguing against contracts. They're awesome. They're just not substitutes for unit tests (or vice versa for that matter). You guys have been arguing against unit tests, which is an absurd position to take, hence this whole debate.

[mempko]

It's not absurd. Unit test are expensive and slow you down while at the same time not targeting the part of software where bugs occur. It's usually HOW functions are composed that cause bugs, not the functions themselves. Contracts rest the interaction of components while unit tests don't.

[sham1]

To be fair, "functions themselves" seldom form a valid unit, so in practice you'd end up having to test the composition of said functions.

[dataflow]

This is definitely absurd... but okay, riddle me this then. Say you're reimplementing string.indexOf(string) with your fancy new algorithm so you can deliver a nice speed boost to users. How do you do your best to minimize the chances of it blowing up on everyone without writing unit tests?

[mempko]

First, users don't use string.indexOf(string). Users use software that uses string.indexOf(string). Second, I would write a test that takes generates a variety of inputs to cover the domain of string.indexOf(string) and have it call the old version and the new version. I would then collect stats around the performance of each call and make sure the new implementation is faster by the threshold I meant for it to be. Since the PURPOSE OF THE REFACTOR WAS PERFORMANCE. Then I would delete the test after I'm satisfied it worked well on the target hardware, etc.

Let's look at the chrome unit tests.

https://chromium.googlesource.com/v8/v8/+/3.0.12.1/test/mjsu...

This test checks one case and then does a loop over to make sure various sizes work. This is a POOR test because it doesn't cover the domain of the function and exceptional cases. This test is pointless to have and run every build. Why keep it, it does nothing useful.

My point is that if you are refactoring a function, you are going to have to write a NEW TEST anyway because the purpose of a refactor is different each time. The old tests are going to be useless. This is why contracts are so useful because they will ALERT YOU when the software does something you didn't assume. Unit tests don't do this.

From an information theory perspective, the old tests would not provide you with new information. If you want to make sure your refactor works like the old function, then compare the outputs of your new function with the old one over the domain of the function. Then if it's good, delete the tests because why keep them.

Experience shows that users will do things to your program you did NOT expect and therefore your tests will not cover. And more likely than not, refactoring requires rewriting tests anyway so the old tests are usually always useless.

Now SYSTEM tests ARE useful. Tests that wire up many components and test them are useful. But a UNIT test, which tests one function is often just a waste of time to keep.