Hacker News new | ask | show | jobs
by christopheraden 4834 days ago
Sample size is something that must be interpreted in the presence of power. You can make a solid conclusion with a very small sample size if the true difference in arrival times is very large, given that the assumptions of the hypothesis test hold (t-test can be a little ridiculous with some of its assumptions sometimes).

In the original article, one of the footnotes mentioned that they tested the data using Wilcoxon's Signed-Rank test, which mitigates a lot of the impact of single outliers.

I'd love to see the raw data though, to see an even less-sensitive method to outliers (sign test). If the difference between the groups is as large as the article would lead us to believe, the loss of power should not present any problem.