[funklute]

Yes, exactly. Control groups are incredibly important for ensuring good quality of clinical studies. It's a technique that solves multiple "calibration" problems, including:

- being able to draw causal conclusions

- being able to adjust against placebo effects

A lot of clever people have tried to come up with ways of doing away with control groups. But ultimately, the best we can achieve is to stop early, as soon as the trial has a clear outcome. I do perhaps think this has become more common in recent times though, so perhaps the study you were involved in was at a time when early stopping wasn't really "the done thing".

But it still beats "studies" done 100 years ago, when you might give someone a cough mixture, see that they improved (if they died, let's just ignore that), and conclude that it was the cough mixture that did it!

[robocat]

> the best we can achieve is to stop early, as soon as the trial has a clear outcome

That must be really hard: if you wait for 95% confidence, you are selecting for 5% noise. If you repeatedly re-measure for 95% confidence, you strongly select for random noise.

Not many medical advancements provide such an anomalously strong signal (98% survival versus 30% survival).

[funklute]

> If you repeatedly re-measure for 95% confidence, you strongly select for random noise.

You can't use standard methods for early stopping - as you rightly point out, you get gibberish if you naively keep peeking at a growing data set. Instead, you have to use statistical methods that explicitly adjust for the repeated sampling in early stopping trials. This does make early stopping more complicated to analyse than a trial with a pre-determined duration.