[adrianmsmith]

I disagree. Unless you are an inexperienced software developer, or it's a completely new domain or technology, or a very experimental problem, you can indeed design the software up-front. Why wouldn't you be able to?

And even if it's an experimental problem or algorithm that's not well understood, and you have to write some test programs, it's still better to spend a day writing test programs and 4 hours doing a design (then throw it away if you encounter a problem with it), than spending 2 weeks actually writing the real code, with documentation, error handling, performance testing, and so on (then throw that away if you encounter a problem.)

[kstenerud]

True, but besides this there are time constraints to any project where money is on the line. You need to get something out quickly that satisfies the market requirements (which usually start off pretty low when you're first entering said market).

So you build the minimum viable product, measure demand (one measure is how many people care enough to complain about your product being so crappy), and then iterate. If you have lots of unit tests, refactoring becomes a less risky proposition, even to the point of re-architecting large chunks of functionality to meet emergent requirements.

Is it slower than building the perfect product up front? In the long run, yes, but in the meantime, you've already tapped a paying market and have gained valuable insight into how people use your product, which guides your further design refinements in ways that cannot be anticipated up-front. And if it turns out the market is dry, you can fail quickly with less resources invested.

That's not to say you should eschew up-front design, but rather the up-front design should be done with the understanding that you're only going for "good enough" with some suppleness built into your architecture for the inevitable refactoring.