[3am]

Test driven development usually refers to unit testing or integration testing. It would be interesting to know if you went beyond that and the nature of contract work you were doing (testing for an internal service with a RESTful API being very different from a mobile app, or a website).

[jacques_chester]

TDD, in my view, starts from the outermost layer -- the end user -- and moves inwards progressively to the unit tests.

[MichaelCrawford]

John Lakos differs from you. In his book "Large Scale C++ Software Development" he advocates starting at the lowest level of your own code, that is, subroutines that only make system calls or call libraries that are provided by the system.

Then you unit test the second level - subroutines that only call that first level, or that call the system or libraries.

main() sits at the top level.

Not every program is straightforward to levelize. His point is that one should do so.

Also the unit tests for any one level should focus on what is new on that specific level, with the assumption that all the lower levels are flawless. In general they won't really be but when that's the case you write a unit test for the lower level.

This keeps the LOC on the tests about the same as the LOC in the deliverable.

It is unfortunate that his book focusses on C++; really he should have written a separate book on testing that was language-agnostic. There is very little of that section that's really specific to C++.

In addition to unit tests I do integration tests. If there's a file format involved I create lots of input files that contain various edge cases.

[geoelectric]

With the caveat I haven't read Lakos' book, there's a pretty big backlash going on in the industry against hyper-granular unit testing. Comes down to people realizing that when their implementation needs to change, they often have to change a bunch of unit tests that were written to the implementation.

One problem is that modularity should let you change implementation without friction; that's the whole point of modularity to begin with. So there's not much profit in writing a bunch of client code (tests) that intentionally break modularity and put friction back on the process. It's not quite as bad as just random breakage, because at least you know where it all is, but it's still painful.

Another problem is that if you're not careful, you end up with a set of tests that don't tell you when something works as expected or not, it just tells you when something changed. Having a test that tells you something isn't coded as expected anymore is pretty useless. You know you changed it.

And still another problem is that the testing itself has become too invasive. We're architecting things for dependency injection that would never really need it if it weren't for invasive tests. It's fine and well to drop some testing hooks in, but if you're having to completely invert control in your code to do it and that makes things significantly more complex, that may not be great.

I think there were some fairly recent Martin Fowler posts on this, but I believe his point was that we're very possibly doing it wrong: if the point is to guarantee an interface then tests should be to the interface. And maybe injecting test doubles is good in some cases but too invasive in others--especially when it's done to test an implementation--and so forth.

So not sure where Lakos is on the scale there, but what you describe about putting in layers upon granular layers of tests strikes me as setting yourself up for these issues. I'm a pretty big believer in testing to the interface, and do try to draw a line between functions that are there to serve a particular implementation and functions that describe something more abstract. I test to the latter.

Modularity is the key to maintenance; the biggest benefit of this type of testing to me is to validate your modularity, even more than validating the code within the modules.

[3am]

Outstanding answer - thanks for the pointer to the Lakos book.