Hacker News new | ask | show | jobs
by TaupeRanger 1740 days ago
"Fetishizing" RCTs (i.e. realizing their importance) is a source of no harm, because it is the best design we have to determine whether some intervention actually helps people live longer or better. Effects from other kinds of studies should of course be weighed accordingly. Fetishizing observational/non-random studies is the source of harm. Taking the Tdap is very unlikely to hurt you, but calling this single study "astonishing" and implying that it indeed shows a 40% reduction in risk (it doesn't) is hyperbolic.
1 comments

If you aren't astonished, you're not paying attention. This study was conducted over six years, on two independent cohorts, involving tens of thousands of patients.

A moment's consideration shows why an RCT to check tobacco smoke and lung cancer was impossible. We are nonetheless 100% confident, in the entire absence of an RCT, that smoking does directly cause lung cancer. The sort of fetishism you promote is exactly what delayed institutional recognition of the fact, by decades.

Ask any professional statistician about failure modes of RCTs. Be prepared to listen for a long time. Instead of worshipping blindly at the altar of RCTs, we should pay attention to what actual statisticians have to say about actual results.

Judea Pearl, in a recent book, "The Book of Why", provides a readable, in-depth exploration of statisticians' fundamental relationship with causuality, and the historical development of statisticians' decades-long loss of their ability to form conclusions from observational data, and their recent recovery. We are all healthier now that we know how and when we may confidently act on results of observational studies, without fetishism.

And yet, due to its design, it is still entirely plausible that there is no, or very little effect. We just don't know.

The reason institutions delayed smoking has nothing to do with RCTs and everything to do with regulatory capture and corruption. Even in the presence of RCTs (which wouldn't be ethical, of course) this still would've happened.

That RCTs have failure modes does not change the fact that they are the best we have in the face of confounding variables - they simply require good design. This claim about Tdap is not like smoking...we don't have the same understanding of the underlying biology nor the massive effect sizes from other studies.

I have read the Book of Why and Probabilistic Reasoning in Intelligent Systems. There is nothing within those books to indicate animus against well designed RCTs. Rather he advocates, as I already did, for intelligent combination of sometimes sparse RCTs with the large N of observational studies.

Your wholesale invention of "animus against well designed RCTs" anyway explains your impassioned defense of fetishizing RCTs. I have never encountered any such animus, and doubt you have, either.

You reveal that you missed that Pearl spells out circumstances where observational studies do suffice to demonstrate what you insist requires an RCT.

And, you show you missed the entire section on the historical events that led to your and others' continuing fetishization of RCTs as the only possible means to demonstrate causation, and their lack to make any such demonstration impossible, which you continue to insist upon, against reason.

Your late admission that RCTs may be poorly designed, and thus fail to demonstrate what they claim, contradicts your previous fetishistic insistence on RCTs' infallibility.