[someguyorother]

>>idea that ideal software development is just a continuous production of small improvements to the code.

>Many of us do believe this. Absolutely.

Isn't Jeffries' attempt to do Sudoku with TDD pretty much the prime example that it's not always so?

Jeffries tried to write a Sudoku solver by starting from tests and incremental improvement, and got nowhere. Norvig knew about constraint propagation and did it in one go.

[jay_kyburz]

I'm not familiar with the story, but it sounds like he started with a solution, (the tests) and tried to find some code that would satisfy them. Perhaps Norvig started with the problem, then tried to find a solution.

[someguyorother]

As I understand it, TDD works by that you say what your problem is, find the smallest thing that will get you there, and then implement a test for it. The test will naturally fail, so you then write code to make it pass, and finally refactor. It's fundamentally iterative.

But Jeffries didn't know how to get to the solution, i.e. what mechanism to make use of to get the computer to automatically solve Sudokus. So he wrote a few tests for the obvious things (like I/O and data structure) and then got stuck.

In contrast, Norvig knew considerably more CS and so knew that the right way to implement a solver is to use constraint propagation. So he did so in about a screen of Python code, all at once, job done.

The lesson of the story, as I read it, is that you can't iterate yourself out of a problem if you don't know how to attack it. If you're making a symbolic calculus program to solve integrals, it's unlikely that you can start off by going "okay, the integral of x is 1/2 x^2..." and somehow, through a series of small steps, end up at the Risch algorithm.

Similar arguments could probably be made about design. The less CRUD-y your application is, the more careful thinking is required. You have to know enough about the domain to know where the pitfalls are, and what the proper tools are.