| To quote Hillel Wayne on LinkedIn, which sums this "study" up nicely: """
Yesterday I read a report claiming that Agile projects had "268% Higher Failure Rates" than projects with more upfront planning. As a fan of upfront planning, and a fan of Empirical Software Engineering, I read the report in more detail. And I'm pleased to report that it is the single worst study I have ever read. It is so bad I do not want to share it for fear of spreading a mind-virus, but I can share some highlights: - The author surveyed 600 people and somehow got P values of 0.00004 - By how he specifically defined "upfront planning", the number of engineers he found practicing it is logically impossible - Every major calculation table had some sort of error - The author edited a whole bunch of Wikipedia articles to promote himself, and claims his results can also cure smoking Today I saw the same report doing on The Register and Slashdot, who both accepted it uncritically. And the hundreds of comments were people accepting the study or suggesting a "confounding factor". ...C'mon, people. You're better than this. I normally write about Empirical Software Engineering because I think using science to study software engineering is important. But just as important is knowing how to protect yourself from bad science. Anybody who's read a couple of research papers would see P<0.00004 as a major, MAJOR red flag. Being in STEM doesn't make us immune to pseudoscience. And we're forever vulnerable to seeing something that confirms our beliefs and not digging deeper to make sure it's not lies.
""" |