Hacker News new | ask | show | jobs
by akoluthic 1740 days ago
This isn't a randomized controlled trial, so I wouldn't put much stock in the results, although it should provide a catalyst for further study (if it hasn't already).
1 comments

Fetishizing RCTs is a source of much harm in medical policy.

A correctly-designed RCT can give more confidence in a result, particularly to non-statisticians, but very large effect and cohort sizes should not be ignored. We could tell just from the topical statistics that smoking was a major cause of lung cancer and other harms, despite the tobacco industry insisting nothing was proven.

Typical RCTs with only a few hundred patients deliver much less resolution than this result. We do still need trials to home in on the molecular agent, but Tdap is already proven safe, and wise to stay current on, so it would be foolish to "wait and see". If it turns out Tdap (e.g.) protects only some patients from dementia, you are anyway safe from tetanus infection.

It is similar to the case where arginine supplements appear to cut epithelial tissue side effects of SARS-2 vaccines. It needs more study, but arginine is cheap and perfectly safe, so starting immediately to administer arginine supplements alongside vaccination is the prudent course.

Some RCTs even yield spurious results, as a consequence of poor design or execution, such as those lately promoted as showing that anti-depressants have no effect.

Fetishism should have no place in medicine.

"Fetishizing" RCTs (i.e. realizing their importance) is a source of no harm, because it is the best design we have to determine whether some intervention actually helps people live longer or better. Effects from other kinds of studies should of course be weighed accordingly. Fetishizing observational/non-random studies is the source of harm. Taking the Tdap is very unlikely to hurt you, but calling this single study "astonishing" and implying that it indeed shows a 40% reduction in risk (it doesn't) is hyperbolic.
If you aren't astonished, you're not paying attention. This study was conducted over six years, on two independent cohorts, involving tens of thousands of patients.

A moment's consideration shows why an RCT to check tobacco smoke and lung cancer was impossible. We are nonetheless 100% confident, in the entire absence of an RCT, that smoking does directly cause lung cancer. The sort of fetishism you promote is exactly what delayed institutional recognition of the fact, by decades.

Ask any professional statistician about failure modes of RCTs. Be prepared to listen for a long time. Instead of worshipping blindly at the altar of RCTs, we should pay attention to what actual statisticians have to say about actual results.

Judea Pearl, in a recent book, "The Book of Why", provides a readable, in-depth exploration of statisticians' fundamental relationship with causuality, and the historical development of statisticians' decades-long loss of their ability to form conclusions from observational data, and their recent recovery. We are all healthier now that we know how and when we may confidently act on results of observational studies, without fetishism.

And yet, due to its design, it is still entirely plausible that there is no, or very little effect. We just don't know.

The reason institutions delayed smoking has nothing to do with RCTs and everything to do with regulatory capture and corruption. Even in the presence of RCTs (which wouldn't be ethical, of course) this still would've happened.

That RCTs have failure modes does not change the fact that they are the best we have in the face of confounding variables - they simply require good design. This claim about Tdap is not like smoking...we don't have the same understanding of the underlying biology nor the massive effect sizes from other studies.

I have read the Book of Why and Probabilistic Reasoning in Intelligent Systems. There is nothing within those books to indicate animus against well designed RCTs. Rather he advocates, as I already did, for intelligent combination of sometimes sparse RCTs with the large N of observational studies.

Your wholesale invention of "animus against well designed RCTs" anyway explains your impassioned defense of fetishizing RCTs. I have never encountered any such animus, and doubt you have, either.

You reveal that you missed that Pearl spells out circumstances where observational studies do suffice to demonstrate what you insist requires an RCT.

And, you show you missed the entire section on the historical events that led to your and others' continuing fetishization of RCTs as the only possible means to demonstrate causation, and their lack to make any such demonstration impossible, which you continue to insist upon, against reason.

Your late admission that RCTs may be poorly designed, and thus fail to demonstrate what they claim, contradicts your previous fetishistic insistence on RCTs' infallibility.