[JoeAltmaier]

"If we control the bug rate, is TDD faster?" is the central question. I assert that TDD is the Cause of the bugs, so the question is nonsense.

Code should be developed with as large a view of the design as possible. TDD restricts development (by definition) to satisfying the tests. The effect is of 'moving the bugs around' since the existing tests rarely completely describe the whole design. You get into a loop of "oh, yeah, well we Really meant to write a test to cover that case too", repeated ad-nausuem.

[ericHosick]

I think one problems with TDD is that it doesn't lean towards testing bugs in a feature/scenario (describing the whole design). That is, TDD focuses on testing a unit and it is effective/fast at testing the fringe cases of units.

Behavior Driven Development really shines in the aspect of "completely describing the design" as it focuses on driving development against a well defined behavior (using scenarios). BDD lets you know where the bug is: in the code or in the specifications (the behavior as specified by the customer). So there is a lot less "chasing around". If you are trying to cover behavior of a system only using TDD then I agree you can end up chasing around a bug.

However, both are necessary and highly effective when used together.

[newt]

I'm not sure what you mean by "TDD is the Cause of the bugs" - the TDD projects had 40-90% fewer bugs. Even if their remaining bugs would have been caught by test coverage of things that occur in live use, that's still a good result, right?

[JoeAltmaier]

Well, sort of. TDD defines bugs as "things the unit tests catch". And the unit tests are very narrow.

So yes, you record fewer bugs, fix those narrow cases quickly and pretend you're done.

Then it gets to the field, and whatever you didn't catch becomes a 3-alarm fire.

I vastly prefer shakedown tests of the sort where you launch a 'bot army' on cloud servers to thrash the app. Especially if there is Any network/client/server component.

[newt]

TDD defines bugs as "things the unit tests catch". And the unit tests are very narrow.

I think that if you read the article linked, you'd find that the study didn't work that way. Any study that measures different things between experiment and control groups is not going to be sound.

Then it gets to the field, and whatever you didn't catch becomes a 3-alarm fire.

I've worked on TDD projects, and those are definitely are counted as bugs. Severity 1.

[steveklabnik]

... and then you get a regression test, so that bug never happens again. And the regression test is easy to write, since all of your code is already testable, you have all the testing infrastructure in place, etc.

[JoeAltmaier]

Never happens again? You already have the 3-alarm fire, its too late. That's my point exactly. You catch only what you can think of and write tests to cover.

[newt]

I'm not sure how TDD makes this worse. Unless you think that it absolves you of any end-to-end QA testing. In which case, welcome to the fire.

[prateekdayal]

I would argue that TDD makes you think more about the design of your system. Also, it makes it easier for you to refactor (as you don't worry too much about accidentally breaking something) and hence keep the code quality high over time

[kerkeslager]

In addition to making you think more about your design, I'd say it makes your design inherently more flexible. Once your design is flexible enough to do two different things, it's much more likely to be able to do three things than if it could only do one thing. And when you do TDD, your design is always capable of two things: fulfilling the requirements and testing.