[baazaa]

A lot of my code takes in garbage data and does some complicated stuff to produce useful outputs. Almost all the difficulty comes from hidden problems in the data upstream, including very often the data contradicting itself (a lot of fuzzy matching, 'voting' to determine the truth, etc.) None of these problems are ever-known beforehand, and can typically only be discovered after a lot of work has been done to make the data more intelligible (i.e. in the process of writing of my code).

I've realised over time that everyone else who prefers the design-doc approach refuse to do work like this. There's just a whole class of problems that are too hard without prototyping.

[softwaredoug]

Right I feel like the problem isn't "prototyping is bad". Prototyping is good! But sadly many companies and devs aren't disciplined enough to throw away first solutions. And that's the real problem.

And its not just the companies, developers get too attached to their first solution, rather than using it as a way to discover knowledge. Companies need to reward knowledge discovery, not just "shipping to prod"