| Dismissing all observational study designs out of hand because they can be difficult and easy to perform poorly seems like quite the take. I see this all the time in people’s interpretation of nutrition research, and they do exactly as this article suggests and fall back to the “intuitive option”, and go onto some woo diet that they eventually give up because they start feeling awful. I would disagree that observational study designs should be thrown out the window or that it makes sense to, as this article seems to do, lump cross-sectional ecological data in with prospective cohort studies. Things often “make intuitive sense” only because of these study designs. We used to get kids to smoke pipes to stave off chest infections because it made “intuitive sense” and it’s only because of observational studies that we now believe smoking causes lung cancer. The direction of evidence from prospective cohort studies to RCTs in the field of nutrition science on intake vs intake shows a 92% agreement. If we take RCTs to be the “gold standard” of evidence that best tracks with reality, it seems a little odd that these deeply flawed observational studies that we should apparently disregard seem to do such a good job coming to the correct conclusions. https://bmcmedicine.biomedcentral.com/articles/10.1186/s1291... |
First, experiments have their own varieties of horrors. Many are small N, with selective data reporting, and lack external validity — that is, the thing you really want to randomize is difficult or impossible to randomize, so researchers randomize something else as a proxy that's not at all the same. Other times there's complex effects that distort the interpretation of the casual pathway implied by the experiment.
Second, sometimes it's important to show that any association exists. There are cases where it's pretty clear an association is non-existent, based on observational data and covariate analysis. You just don't hear about those because people stop talking about them because of the null effects. So there's a kind of survivorship bias in the way results are discussed, especially in the popular literature.
It's easy to handwave about limitations of studies, it's much harder to create studies that provide evidence, for logical, practical, and ethical reasons. Why you'd want less information about an important phenomenon isn't clear to me.