[funklute]

> Everyone knew

If that was truly the case, then - as the parent hinted - it was the trial design that was the problem, and not the practice of having control groups.

It is pretty common to stop trials early when it is obvious that the treatment works. Conversely, it is easy to think that "yea of course the new treatment works" despite the evidence not being there. The need for robust analysis must also be respected.

[protomyth]

It was my understanding that every study has a control group getting the standard care. I didn't see a study without such a control group.

[funklute]

Yes, exactly. Control groups are incredibly important for ensuring good quality of clinical studies. It's a technique that solves multiple "calibration" problems, including:

- being able to draw causal conclusions

- being able to adjust against placebo effects

A lot of clever people have tried to come up with ways of doing away with control groups. But ultimately, the best we can achieve is to stop early, as soon as the trial has a clear outcome. I do perhaps think this has become more common in recent times though, so perhaps the study you were involved in was at a time when early stopping wasn't really "the done thing".

But it still beats "studies" done 100 years ago, when you might give someone a cough mixture, see that they improved (if they died, let's just ignore that), and conclude that it was the cough mixture that did it!

[robocat]

> the best we can achieve is to stop early, as soon as the trial has a clear outcome

That must be really hard: if you wait for 95% confidence, you are selecting for 5% noise. If you repeatedly re-measure for 95% confidence, you strongly select for random noise.

Not many medical advancements provide such an anomalously strong signal (98% survival versus 30% survival).

[funklute]

> If you repeatedly re-measure for 95% confidence, you strongly select for random noise.

You can't use standard methods for early stopping - as you rightly point out, you get gibberish if you naively keep peeking at a growing data set. Instead, you have to use statistical methods that explicitly adjust for the repeated sampling in early stopping trials. This does make early stopping more complicated to analyse than a trial with a pre-determined duration.

[anthuswilliams]

No, in fact it is quite common to do "open label" clinical studies in which patients know what drug they are getting and no one is given the current standard of care. This is especially common in cancer, where the standard of care is so poor, and in rare diseases, where the patient population isn't large enough to admit such a comprehensive study.

It is harder to have the same statistical confidence of efficacy and safety in such studies, but clinical researchers try to address those issues by varying e.g. dosage quantities, time in between doses, etc.

Source: place I work is currently doing studies of this nature, and in general such studies seem to be well-understood and accepted by the FDA.

[s1artibartfast]

Randomization is independent from blinding. Open label studies can be either randomized or not, and controlled or not.

The FDA will usually push for blinding if possible (sometimes it is not). They will also usually push for a randomization zed control with standard of care. They way the FDA views it (and I agree), is that control patients are not harmed because they are still getting the same care they would outside of the trial.

It is usually unethical to have a treatment free arm. However, this has its own problem, where if you keep using equivalence comparisons, the end of the chain might not actually be any better

[tsbischof]

For such a large effect it is quite possible to implement an adaptive trial design with unbalanced arms and interim analysis for efficacy. But you have to ask for this, the FDA will not necessarily suggest it directly.