Hacker News new | ask | show | jobs
by bildung 3924 days ago
> No, this is wrong. With small sample sizes you may get a statistically significant result, but it still might not be a real result and might not be reproducible. This is a major issue in science today and why a lot of studies can't be replicated.

Reproducability indeed is a major problem, but looking at statistical significance alone isn't the cure (especially if applied a posterior).

We should rather look at effect sizes and robust study designs.

In fact, modern studies aiming for causality often calculate the population size needed for statistical significance beforehand. It's a standard formula in most textbooks. You only need the expected effect size and then can calculate the population needed to guarantee significance.