[Rapzid]

How many tests does a PR need? One? Five? When would you not write tests for something(that something not necessarily being a PR, but maybe a unit or feature)?

Tests, like any process, should be serving you and your goals.. You shouldn't be serving your processes or testing practices. This sort of un-nuanced thinking isn't indicative of a high performing startup or CTO IMHO. Perhaps your policies are not directly indicative of your real thoughts on the matter?

[ereyes01]

I guess it might sometimes be fine to be a relativist and write off the need for tests as a result of "nuanced thinking," but I think you have to accept that you are running a risk by shipping untested code into your product.

As others have said, line coverage is a misleading metric. Ideally, your tests would fully cover all _program states_, and even 100% line coverage doesn't guarantee full state coverage. If you have untested states, then the following facts are true:

- You don't have a formalized way of modeling or understanding what will happen to your program when it enters an untested state.

- You have no way to detect when a different change causes that state to behave in an undesired way.

So the answer to how many tests does a PR needs- as many as needed to reduce your software's risk of failure to a minimal level... And this is failure right now, and in the future, because you will likely be stuck with this code for a while. Since it's difficult to know how much a future failure will cost your company, IMO I always try to err on testing as much as possible. Plus, good comprehensive tests have other benefits, such as making other changes / cleanups safer by reducing the risk that they unintentionally side-effect other code.

[kod]

Those facts are untrue. If i am using a sound static type system, I have a formal way of modeling and understanding what will happen to my program, even without tests.

If a function has been statically proven to return an int, I know it will either return an int or not return at all. It can't suddenly return a hashmap at runtime, no matter what untested state it enters.

[auganov]

Code without test code doesn't mean untested code. And vice-versa.

Unless you're actually writing complex tools - no, you're probably not getting a "formalized way of modeling" what happens to your program.

If somebody tells me "hey, I have to keep manually testing this and that, I'm losing a lot of time, how about I spend 2 days writing my test thing?" - I'll say Sure!

But if someone tries to convince me in the abstract - I'll be skeptical. Developer busy-work is real.

[njs12345]

If you have any concurrency in your system then you aren't going to cover all the states using unit tests. You'll need some sort of formal model for that.

[arkh]

> How many tests does a PR need?

Enough test for each of your spec. Adding a new functionality to your product? Your test should cover and the cases you put in your specs. Correcting a bug? You test should trigger it with the old code.

You can have 100% code coverage with unit testing, it will do jack-shit for your users when they enter a negative number and there was no code meant to manage this case so it never was tested.

[aianus]

> How many tests does a PR need? One? Five?

Enough so the overall coverage doesn't go down.

[ghayes]

Be careful that coverage is a proxy metric to good tests. Striving for high coverage can mislead you on the quality of your tests.

[rootusrootus]

On the flip side, encouraging good coverage usually ends up uncovering some bugs that might otherwise have gone unnoticed until they bit someone.

[Mikushi]

On the other flip side it encourages writing test that have zero business value.

[aianus]

High coverage is necessary but not sufficient, sure. I don't think you can have a good test suite with low coverage (< 80%).