| > This isn't correct. The statistical power of n=5 humans is quite low. A few points are important to consider. First, I was only talking about experimental studies searching causal relationships. There are other possible designs, for example field studies (e.g. "school district A gets the new math curriculum, school district B the old one. Which one fares better?") or simple population observations ("people playing golf live longer than the average population."). Each design has advantages and disadvantages regarding generality of the statement one can make, and for each one different statistical considerations apply. Second, the statistical power does not rely on a high population alone, as that (more or less) only affects the significance tests. Much more important is the effect size. If you can measure a large effect (as this study did), it's pretty hard not to reach significance anyway. Third, from a statistical point of view, the population isn't 5, but much higher. Let me explain:
There are certain kinds of treatments whose effect is reversable. Caffeine intake is an good example: Once you stop taking caffeine, the effect recedes. While designing the study, you can use that property. One common way is an ABAB design, where A is a phase with treatment and B is a phase without. You can chain as much AB pairs as time permits, and additionally you can measure multiple times per phase. Statistically, the population now is real_humans x number_of_phases x measure_points_per_phase. |