Hacker News new | ask | show | jobs
by newt 5671 days ago
I'm not sure what you mean by "TDD is the Cause of the bugs" - the TDD projects had 40-90% fewer bugs. Even if their remaining bugs would have been caught by test coverage of things that occur in live use, that's still a good result, right?
1 comments
JoeAltmaier 5671 days ago
Well, sort of. TDD defines bugs as "things the unit tests catch". And the unit tests are very narrow.

So yes, you record fewer bugs, fix those narrow cases quickly and pretend you're done.

Then it gets to the field, and whatever you didn't catch becomes a 3-alarm fire.

I vastly prefer shakedown tests of the sort where you launch a 'bot army' on cloud servers to thrash the app. Especially if there is Any network/client/server component.

link
newt 5671 days ago
TDD defines bugs as "things the unit tests catch". And the unit tests are very narrow.

I think that if you read the article linked, you'd find that the study didn't work that way. Any study that measures different things between experiment and control groups is not going to be sound.

Then it gets to the field, and whatever you didn't catch becomes a 3-alarm fire.

I've worked on TDD projects, and those are definitely are counted as bugs. Severity 1.

link
steveklabnik 5671 days ago
... and then you get a regression test, so that bug never happens again. And the regression test is easy to write, since all of your code is already testable, you have all the testing infrastructure in place, etc.
link
JoeAltmaier 5671 days ago
Never happens again? You already have the 3-alarm fire, its too late. That's my point exactly. You catch only what you can think of and write tests to cover.
link
newt 5671 days ago
I'm not sure how TDD makes this worse. Unless you think that it absolves you of any end-to-end QA testing. In which case, welcome to the fire.
link
JoeAltmaier 5670 days ago
I object to the TDD fundamental precept that you code to the tests. That narrows the 'testing' until its a self-fulfilling prophecy that you will have 'fewer bugs', since you defined the bugs as 'what we're looking for'.

Reminds me of the old joke "A: I'm looking for my wallet. B: where did you lose it? A: over there. B: then why look here? A: the light's better"

link