Hacker News new | ask | show | jobs
by bOR_ 6022 days ago
This sample-size too small thing is becoming ridiculous. You do realize that although something might not be significant, in a hundreds of situations in real life, you are perfectly willing to draw real conclusions from a sample-size of 1?

Stupid example: You are with a group of people, and wonder if it is safe to swim in this river. The first person that walks in gets grabbed by a crocodile. Do you ridicule the rest of the group for not going for a swim? Especially since the only crocodile you observed will not be needing another meal.

1 comments

Of course not—the potential downside from getting that wrong is getting eaten by a crocodile, and the potential upside is that I get to swim. I demand much more confidence from my decision making process because the risk-reward is so different. Further if I update my priors in a Bayesian manner the fact that of the sample size of 1 person going into the river, 1 person got eaten is somewhat worrying. But because you're not giving me any information, my prior is impossible to calculate. If this is a river in California I would be rather surprised to see a crocodile. If this was the Nile delta I would not. Etc.

Here the downside to getting things wrong is much smaller, so it's much easier to hold out for more precise data before making any claims.