[mosselman]

I hear this argument a lot, but I do not think it is valid. What happens is that you simply put in the time before a bug occurs with every unit test that you write. Which isn't very efficient, seeing as not all of your units will lead to bugs. So yes, once you have all of your Units covered 100% it will be easier to find bugs, but you have invested a lot of time in order to get here and in order to keep the unit-test-suite maintained.

[pron]

But the goal of tests isn't just to find bugs in newly-written code. It's to defend against regressions, and a regression is even harder to localize, given the team may not even be well-familiar with the failing code.

[bunderbunder]

This is a spot where the potential for test-induced design damage comes into play.

With bite-size integration tests, I find it's generally not too hard to isolate the cause of a failing test, because the code it's testing tends to be straightforward, and fairly easy to step through, if necessary.

I frequently have a harder time with unit tests. The code ends up involving a lot of extraneous abstractions that I need to think through. The test code is often so heavily mocked that it's hard to distinguish the behavior under test from stuff that's just being mocked or stubbed to get the SUT to run cleanly, meaning I've got to start with trying to figure out whether the bug is in the test or the code being tested.

It gets worse in long-lived code bases, where the unit tests are often subject to significant bit rot on account of how brittle they are. I've definitely had some code archaeology excursions reveal that the reason an entire suite of tests were tautological is because someone was doing a nominally unrelated refactor, and just put in the minimum effort necessary to get the tests to go green again.

You can argue that developers need to be more diligent. Me, though, I figure it's sort of like those lines of bare dirt you see criscrossing the lawns of university campuses: when things get to that point, it's a sign that the official way of getting around isn't appropriate to most people's real needs.

[bunderbunder]

I should say, I was complaining there of code that is pervasively unit tested, not unit tests in general.

I do think it's important to have unit tests when the unit's behavior is complicated. Where I start to get worried is when there are unit tests being written against classes that have very little behavior that doesn't involve interaction with some other module.

[394549]

> I do think it's important to have unit tests when the unit's behavior is complicated. Where I start to get worried is when there are unit tests being written against classes that have very little behavior that doesn't involve interaction with some other module.

I think this is precisely where unit test suits start to have problems. Good, flexible unit testing requires a lot of judgement about what will be useful to test and what will be too much of a burden in the future. Unfortunately judgement is hard to acquire and even more difficult to teach, and a lot of teams want to create and enforce over-dogmatic testing "standards." When unit testing, you have to balance:

1. What testing do I need to have confidence my code is working?

2. What testing do I need to catch likely regressions?

3. What kinds of tests will just get in my way in the future or are literally useless [1]?

[1] E.g. unit tests that essentially only test core language functionality, once you take out all the mocks.

[ThorinJacobs]

This is a really important aspect of good testing that doesn't seem to get as much attention as it deserves - the tests that have brought the most value for me are the ones that assert the business requirements, not the implementation details.

So for a little passthrough/orchestration class, it probably doesn't make sense to do much testing. For something that actually performs business logic, that's a prime candidate for testing. I've seen plenty of tests that just seem to aim to increase coverage, heck, I've written plenty of those myself - but at the end of the day, the benefit they serve after being written is probably minimal.

[toast_coder]

Unit test that cover complex regression scenarios are difficult to identify up front and are missed frequently.

[Roofduck]

I agree that regression scenarios are difficult to identify, this is why its good to have unit tests to begin with.

You're only unit testing the code how you 'intended' for it to work at that time. Even though the tests are written, it probably wouldn't be uncommon for a bug to slip through when running your code, what you then can do is write another test to account for that scenario, then repeat and your code becomes more robust as a result.

[bluGill]

This is both correct and incorrect.

Correct that the goal is to prevent regressions. I claim (I don't know how to study this) 80% of your tests will never fail and so they could safely be deleted - but I have no insight into which tests will fail so I say keep them all.

Incorrect because in fact it isn't hard to localize failures: it is something in the code you just touched!

[pron]

> it is something in the code you just touched!

Yes, but you also need to localize the effect of the bug to know why is it that the code you changed broke the program (and remember that we're talking about a case where the rest of the program is not familiar to you enough). Good unit tests can help you find the immediate effect of the failure, rather than the ultimate one.

[abecedarius]

I don't understand how you could live the experience that the problem is always with the just-edited code. The closest I can come is supposing that you've always worked with thoroughly unit-tested code (and correct, well-documented libraries, etc.).

[bluGill]

I'm lucky enough that the code base I work on was a "big rewrite" not long enough with the goal of having everything tested. Also we are an embedded system where we know exactly which version of each library to support and upgrading any library is in itself a big deal done as a separate exercise.

[ams6110]

Maybe I'm a worse than average coder, but I find it nearly impossible to write a "unit" of code without also writing one or two bugs along the way. So whether you do your testing ad hoc as you develop, or write unit tests, you do have to spend time in testing each unit of code you write. It doesn't seem to me that formalizing this into a repeatable unit test is adding a lot of real extra work.

[Retric]

I would suggest trying to write correct code without unit tests. They fill up mental capacity making it harder to think about then write correct code in the first place.

Keeping below 1 bug per 100 lines of code is viable simply by being careful and thinking things through. That's a long way from perfection, but it really helps.

[dpark]

Advice that boils down to “try harder” typically provides no value. If you want an outcome to change, you have to change an input. Better tooling or better processes may achieve that. Trying harder rarely will, because most people are already trying quite hard to do a good job.

[Retric]

I think people may be reading this backwards. I am specifically saying don't try as hard. Don't think about unit tests or anything else when writing correct code for a function.

Once you can generally write mostly correct code then you can work on improving your process. But, until most of your functions are working when you right them just work on improving that.

Edit: And yea quality may drop as part of this transition, but you need to get a feel for how much you can pay attention to.

[dpark]

You’re saying that if you have a problem with code correctness, the solution is to just try harder to write correct code.

The tests are there specifically to help find the errors that “trying harder” didn’t catch. You don’t get a higher quality result by cutting QA.

[Retric]

I don't necessarily think he has a problem with code correctness after QA. I think he has a problem with code correctness before QA.

You can build a factory that has a Great QA process that finds all faults and corrects them. But, inside the factory you still want to minimize the amount of defects for QA to find. Either way you still need QA, but it's easy to get into the habit of improving QA vs improving the initial process.

PS: One exercise I did for a while was every time I found a non syntax bug in a function I just wrote (aka bug at run-time) I would start over and rewrite it. It's painful, but 'Build' wait let's look at this again if I hit run and it does not work that's going to be painful. So is this actually correct? Anyway, doing that showed me how important it was to just focus on the code (and what it means) to exclusion of everything else.

[curun1r]

> until most of your functions are working when you right them

Heh...irony alert. You're advocating focusing on just getting it right the first time and you didn't get write right.

But seriously, the people you're responding to are correct. Any process that relies on humans being less error proneis bound to fail. You need to either create a process that makes humans less error prone (e.g. checklists) or embrace our propensity for making errors and plan for that eventuality.

[blktiger]

I find that writing unit tests helps me think about the code so that I write better code. Plus it generally forces me to make the code more modular than I might otherwise. YMMV

[Retric]

If it's working for you then great. I am specifically responding to:

> I find it nearly impossible to write a "unit" of code without also writing one or two bugs along the way

Finding bugs early is great, but minimizing bug creation is even better.

[verinus]

THen why not think "in-code"? I mean thinking through all aspects of edge cases and the like would likely result in some written documentation- why not do it in code then?

[Retric]

You can only focus on a relatively small number of different things at the exact same time.

  1 3 2 5 9 2 8 7

vs.

  1 3 2 5

  9 2 8 7

Memorizing the first sequence is trivial, but dealing with each half independently is much less mental effort. Now each idea can be more complex than just a number, but even still you make fewer mistakes by removing mental overhead.

PS: Now, their are a lot of tricks on how you can get better at this stuff. But if the parent poster tends to write bugs in most functions then simplifying things may help.

[thephyber]

I have found unit tests are useful for the original developer to very explicitly describe what test cases the code was written for (including parameter data types). The lack of unit tests for specific types of use cases can also signal to the inheriting developer what the code wasn't originally expected to do (which can signal performance limitations).

[Klathmon]

It's a trade off of when you want to spend the time.

Sure, it's less efficient to write unit tests ahead of time for every case, but in a lot of cases fixing a bug faster when it's discovered is much more important than even double the hours spent during less "pressing" times.

[skohan]

But I think the argument that the article is making is that it's not only the case that you're moving the time spent fixing bugs, (i.e. a greedy vs lazy strategy), the issue is also that the baseline maintenance cost of the code base is increased by orders of magnitude when unit tests are considered.

I tend to agree - in my personal experience in organizations with strict TDD culture, a perverse incentive often emerges to preserve existing flawed architecture over obviously better solutions just because it's so painful to deal with all the tests.

One of software development's most powerful properties is the ability to iterate quickly: it's foolish to prioritize dogmatic beliefs about testing over that quality.

[Klathmon]

Yeah I absolutely agree that in many cases tests become more of burden than a help, but I think that isn't a problem with testing but with it's application.

It's just an extension of your code, if you are going to throw some code away to change an interface, then throw the tests away too. If you are afraid to do that because of the time spent, then you probably spent too much time on writing tests.

While I am dogmatic about tests, I also believe that around 50% code coverage is normally enough in most application codebases. Cover the important parts, the parts that are hard to test manually, the "core" pieces that lots of other areas rely on, and some tests for bugs that you want to prevent happening again.

If you want to quickly iterate, go for it! You shouldn't have all that many tests in the parts you are changing frequently. But to change a core aspect of the codebase, or a really complicated aspect of it, then the extra work of rewriting the tests shouldn't be all that bad.

I'd shorten your last statement. It's foolish to prioritize dogmatic beliefs. Everything has a time and a place, and moderation is key.

[eeZah7Ux]

> Units covered 100%

This metric means very little. It does not measure the extent of code path coverage and much more.

[rhencke]

There is only one thing I have found code coverage is good for.

Code coverage can tell you what code is not tested at all.

Now, this is very useful. But:

Code coverage can't tell you what code _is tested_.

Code coverage can't tell you how _well_ code is tested.

[ravenstine]

It's also a good way to find parts of the code that are not being used and are worth deleting.

[mosselman]

Yes that is the essence of what I was saying.