| > RCTs are also observational The key phrase is "purely observational". Now, occasionally you end up with a "natural experiment" in which some accident has effectively done the randomization for you—specifically, where the mechanism that puts people in the treatment group vs the control group is something you can be very confident has no other causal interactions. This was a good example: https://twitter.com/PGeldsetzer1/status/1661776663074738176 "Causal evidence that herpes zoster vaccination prevents a proportion of dementia cases [...] To provide causal as opposed to merely correlational evidence on this question, we take advantage of the fact that in Wales eligibility for the herpes zoster vaccine (Zostavax) for shingles prevention was determined based on an individual's exact date of birth. Those born before September 2 1933 were ineligible and remained ineligible for life, while those born on or after September 2 1933 were eligible to receive the vaccine." But any time you're looking at a scenario where treatment vs non-treatment was the result of individual human choices, that opens up a potentially very wide range of ways for something you didn't know about (and potentially something difficult to accurately control for even if you do know about it) to cause treatment and cause the outcome, instead of the treatment causing the outcome. > it suggests that we can't build confident causal inferences without a certain study I do think there's an upper limit to the confidence you can justifiably hold, and that it's often not very high. Consider the studies that observe "A bit of alcohol correlates with better health than zero alcohol". You control for wealth, education, and maybe other things, and the apparent effect remains. How high confidence should you have in the result? Then someone realizes: Some fraction of people who consume zero alcohol do so under doctor's orders because they have health problems, and if you exclude those people then the effect disappears. No matter how many causal pathways you think you've controlled for, how confident can you really be that there isn't a new one you haven't thought of? (And controlling has its own perils: if your measurements are noisy, or if you end up controlling for the outcome.) > it's incorrect to say they can uniquely detect causation Oh, correlational studies can "detect" causation, but the hard part is being certain that the thing detected isn't a false positive. |