[agentultra]

TDD does not do anything to ensure the API is good.

Well, what is good? You suggest such an API has to solve a complex problem with hundreds of users. I should think that I've written a few modules with just such an API using TDD methods. So what are we really saying here with this definition?

Your elaboration goes back to the myth I would like to see dispelled as my experience, and that of many others, has demonstrated that it's a bit of a red herring.

When I start a module I don't know what the API should be or what invariant is important or anything. All that test driven development asks is that I first make an assertion about my problem and try to prove that it's true (or not as the case may be).

This leads me to good API designs because the tests invariably contain the assertions about my invariant, they document the API at a high level, and they demonstrate that my implementation is correct as to those assertions I made. If I am thoughtful in this process I know that some assertions contain a quantification over a universe of inputs so I use property based tests to drive my specification. Other times a handful of examples is good enough to demonstrate correctness and so I simply test the units with them. And the API is entirely under my control this whole time. The tests are guiding me to a good API that does exactly what I say it will do by way of executable specifications that cannot be wrong about my implementation (or else the test would fail).

There's nothing in the above that says I cannot take a top-down approach and design my module's interfaces first. Nothing.

Personally I prefer the bottom-up approach for many of the reasons I explained in my OP. I like to define things in small units that I can compose over with abstractions that let me achieve more interesting results. Testing keeps me honest as to the true requirements so that I don't go off into the woods.

[bluGill]

Good is in the eye of the beholder. And I was intentionally implying hundreds of users. If - this is the majority of APIs - there are only a couple users it isn't worth the effort to make the API good. When there are hundreds or thousands of users it becomes worth the time to start at a whiteboard and design the API, a few hours up front now can save minutes in the future and those minutes add up.

What you are missing is that TDD gets you to AN answer, but it gives you no information that you got the best answer. TDD does make some bad design choices hard or obvious; but there are other times where that there was a better choice isn't obvious.

Bottom up and top down are very different design considerations. You can get bad APIs in either one. You can do TDD with either.

I'm a fan of TDD and use it. However I'm not blind to the limits.

[agentultra]

If what you're saying is that TDD can guide you to a good design but it doesn't guarantee it then I think we're in agreement there.

When working on protocol specifications and proving the correctness of a certain invariant or property of the system I find TDD to be lacking and tend to turn to formal methods for results.

However at the module/interface level of an individual software library or component TDD is a very good tool that helps guide me towards a good design. As do other tools like a sound type system. When they work in concert the system almost writes itself -- I just have to come up with the queries, invariants, and assertions.