|
|
|
|
|
|
by icebraining
4230 days ago
|
|
|
It's not just a meme. There have been multiple articles and papers the last few years on how a great number of clinical trials are statistically underpowered and therefore suspect; this includes a 2005 paper entitled "Why Most Published Research Findings Are False", which analyzes "49 of the most highly regarded research findings in medicine over the previous 13 years" and finds that a worrying number of them hasn't stand replication. And when people talk about underpowered studies, it's mentioned less than 100 individuals, not less than 20. |
|
|
Small samples get overinterpreted, many published findings are false, etc.—it's all true and the community here is well aware. But it's a gross overreaction to dismiss small-sample experiments wholesale. Even a few seconds' reflection is enough to see that.
For example, assuming this experiment was rigorous, it didn't need more than 16 subjects to find that greater saturated fat in diet doesn't automatically cause greater fat in blood. Additional resources might be better spent on future samples (i.e. replicating the finding by other researchers) than on a larger single sample, which is probably a game of rapidly diminishing returns. And so on.
The point is that HN wants reflective discussions, not reflexive ones. It takes no work and no real thought to pick out one detail that people are currently primed to fuss over and make a post of it. That's not reflection, it's habit, and its payload is not learning, but reinforcement. Reflection requires engaging with the material—this specific material.
There are two ways to do that. One is to dig in and learn the material, think about it, and report your findings to HN. The other is to happen to know something about it in the first place. The first takes work, the second luck. Comments based on neither work nor knowledge are likely not to be substantive. That's why want to avoid generic dismissals as opposed to specific ones.