[noelwelsh]

I think the article goes a bit too far against data. Hits like Bohemian Rhapsody are by their nature freak events and not easily reproducible. Nobody is suggesting (I hope) that you can achieve that kind of success without a healthy dose of luck.

However I agree that for early stage startups data driven decision making can be difficult. My experience is its expensive and you often have very little data.

The other issue is this kinda uncanny valley of false rigor. On the one extreme you have very informal analysis. For example, we tweaked our blog post template and increased newsletter signup rates but I can't tell you exact %s because at this stage we don't track it. We seem to be getting a lot more signups, but perhaps its illusory. That's ok. At our level of traffic it really isn't important. At the other extreme you try to model non-stationary processes and all that and have rigorous control over sources of error. In the middle is where I see many companies with, say, A/B testing, believing they have a high level of statistical rigor but not actually achieving that rigor in practice due to many uncontrolled sources of error. This middle spot, where you have too much faith in faulty reasoning, is where I believe bad data driven decision making resides.

Oh, and on turd polishing: http://www.dorodango.com/create.html

[frandroid]

> Hits like Bohemian Rhapsody are by their nature freak events and not easily reproducible.

Just like startups...

[collyw]

Reading the article it reminded me of when the Morotola Razr came out. Before that all the data driven studies had been done, and the ideal handset size and shape had been determined. The Razr went against all the rules (it was far too wide according to these studies) and was a huge success at the time.