| (Note: They don't link to the study in the article so naturally I cannot comment on the soundness of the study in question). (EDIT After writing this comment the link was changed from the Guardian to the NYT, which provides more information, though the study has still not been published.) Studies (especially psychological ones) with human subjects are very difficult and expensive to conduct, which is why sample sizes are often small. 50 is by no means an unusual size for an experiment of this sort. An alternative is to use observational data, but it's very hard to differentiate between useful observational data and garbage observational data. Not only do you introduce a whole host of problems, but it's more difficult to parametrize the problems that are introduced. So you could easily create a study that boasts a large sample size, with a respectable p-value[0], but have no way of knowing which confounding variables were introduced during the data collection process. A third option is self-reported data, which comes with an even bigger asterisk after it. For something like this, I'd much rather trust a controlled study of 50 than a self-reported survey of 300 (at that point you might as well post it as an 'Ask HN' and judge based on the comments!). By contrast, while controlled experiments on human subjects are by no means unbiased or immune to confounding of variables during data collection, it's almost always easier either to limit these in a controlled setting or at least to parametrize them after they happen. So in the end, this usually ends up being the best feasible option (not the best (theoretically) possible option), short of massively increasing funding to such studies. [0] Which is usually the wrong way to look at studies anyway, but that's a separate topic of discussion |