Hacker News new | ask | show | jobs
by paviva 1633 days ago
There is no point.

The fragility index is essentially a repackaged p-value, and serves only to introduce even more confusion on the top of this already often misunderstood metric. See very good discussion here: https://academic.oup.com/eurheartj/article/38/5/346/2422087

Also, trials are generally designed to recruit the minimal number of patients we can get away with, and this is done for good reason (cost, feasibility, and the ethical concern of "using" the lowest possible number of human beings, while allowing any beneficial treatment to benefit the most as soon as possible). If someone's then pointing out that this trial is "fragile" -- well, yes, it was designed so in advance! Would you propose to expose 100 more patients to an inferior treatment just to improve your fragility index?

1 comments

Honestly? Yes. The issue I'd have with said minimums is that, at least from my experience in biomedical research is that they're often done with exceedingly poor statistical grounding and pathetic sample sizes. Admittedly, perhaps things are better in the clinical space, though my gut argues otherwise.

So, to re-answer your question

>Would you propose to expose 100 more patients to an inferior treatment just to improve your fragility index

Absolutely, especially if the alternative is exposing a million more patients to a different, more expensive inferior treatment because the statistical analysis was garbage.

Garbage statistics will not get better if you insist on some arbitrary fragility index threshold (and no accepted threshold of an "acceptable" fragility index does exist to the best of my knowledge). Also, using a lower alpha will automatically give you a larger sample size, without invoking a completely superfluous fragility index.

Nobody should interpret clinical studies in isolation; they only have meaning in a "qualitative" Bayesian framework which integrates physiological plausibility, other available trials, and risk/benefit ratio. The fragility index only muddies waters, as clinicians misinterpret it even more frequently than the much maligned p-value, all while not delivering any more information than the p-value itself.

Yeah, Full agreement on the arbitrary fragility index. One of the main issues with the more squishy sciences is the cargo-culting around specific breakpoints, which happens due to a lack of statistical chops to begin with.

At least at the labs I've been at, the vast majority have no clue what those p-values they get mean, nor do they realize that the 0.05 cutoff that they so often target is entirely arbitrary.