[hexl]

> but without a randomized control trial, there is no way the success can reliably be attributed to the medications

That's non-sense, if everyone that had cancer got cured by a miracle drug, would you still demand a randomized control trial?

[gus_massa]

Yes! I want to be sure that this miracle drug is 100% to eliminate all the other drugs that has nasty side effects and have a lower effectivity.

Let's pick some terminal ill patients. I think cancer metastasis in the brain has a very bad prognosis, like less than a year of life expectancy [1]. Let's make a one year trial: 100 patients in the control group that receive the usual treatment and a placebo. 100 patients in the treatment group that receive the miracle drug instead of the placebo. Obviously a double blind study.

If the terminal patients selection was good enough, after a year you will get 90 death in the control group (there are some lucky guys) and only 5 death in the treatment group (someone died in a car accident). That would be very convincing and in a few years (with a few additional studies) it will remove all the current drugs from the market.

Without a serious study, some doctors will believe in the miracle drug and some will have the gut feeling that another drug or drug combination is better and continue using the old treatment. The lack of a convincing study kill people.

And also, there is the risk of snake oil. Some doctors are convinced that a drug cures 100% of the patients and push it to be applied to everyone. Sometimes they are wrong, without a study it is impossible to separate the good and the bad ideas. The lack of a convincing study kill people.

[1] Unless it a metastasis of breast cancer and is affected by hormones? I think there a few exceptions, but it is usually very bad.

[Dylan16807]

So you have a cancer that normally has a 90% fatality rate after a year. Someone gives you evidence that they gave this drug to everyone with this cancer in their hospital, and 70/73 survived for a year. Not properly randomized, not properly controlled. Would you really reject their data and find it unconvincing?

[gus_massa]

If the results are so good it would be impressive. But it will be necessary to take a look at the data. Dollar to doughnuts they have a horrible methodological mistake, like a bad classification of the patients, or using a weird definition of cure [1], or cherrypicking the patients that get cured.

[1] We have a big announcement of a miracle cure in Argentina in 1986. It was crotoxina [links bellow] that is a part of the venom of some snakes. One of the problems was that they were comparing CT from different angles and finding fake reductions in the size of the tumors. (Other parts of the study were just frauds.)

https://translate.google.com/translate?sl=auto&tl=en&u=https...

https://translate.google.com/translate?hl=&sl=es&tl=en&u=htt...

https://translate.google.com/translate?hl=&sl=es&tl=en&u=htt...

[Dylan16807]

Valid issues, but you can have all of those problems in a randomized controlled trial too. That's not a very compelling argument that the non-random poorly-controlled version is any worse in this hypothetical.

[gus_massa]

For example if the "usual" death rate is 90% and you select some group of patients for security reasons, like 18<=age<=60, perhaps the death rate is reduced to 80%. If the testing and control group is randomized, then if there is no effect of the drug you will get the same 80% in both groups. If the group is not randomized, you can't be sure that any of the selection criteria is the cause of the improvement.

[Dylan16807]

You can look at the selection criteria and perform an analysis. If the data is clear enough, you can still pull a signal out of the noise. I assume that's the reason you took a 70/73 survival rate in the hypothetical I posed and reduced it to 20% in your version. And to be clear, in my version it's a general hospital and they give the treatment to everyone that has this diagnosis. There are no intentional selection criteria, it's mostly just who lives in the area.

[Fjolsvith]

In this case, it is a bit easier to deduce a successful result, in that the patient survives death versus having a tumor shrink in size.

[Fjolsvith]

Unfortunately, you cannot correct nasty side effects for patients who die from lack of treatment.

[gus_massa]

Note that some cancers kill you very fast and other kill you in a very long time, so you have a chance of die from another cause.

If the study in a small group proves that the miracle drug is ineffective and moreover it reduces the life expectancy to 1/2, then by not giving it to everyone avoid the nasty effect of reducing the life expectancy of a lot of people.

Take a look at some "miracle" cures of cancer of the past, like https://theincidentaleconomist.com/wordpress/the-rise-and-fa...

Also, since a few years in Prostate Cancer the recommendation is not to treat all cases after detection https://en.wikipedia.org/wiki/Prostate_cancer#Management

[Fjolsvith]

Of course, this would apply especially to a drug that hasn't been in regular use for 60 plus years.

Side effects of HCQ are well known, and safe dosage has long been established. If it saves some lives, why deny it to patients knocking on death's door?

[h_r]

It's simply a problem of what is considered to be evidence in the scientific community. If everyone "got cured by a miracle drug", that implies you already have the compelling evidence in hand that the drug, in fact, is a cure. This is an example of begging the question.

[Dylan16807]

It's not begging the question at all! The entire point is that you can have compelling evidence without a randomized controlled trial.

[h_r]

Again, this is the very thing that needs to be shown. You're simply repeating the same mistake.

The post I was responding to really had no _point_, per se. It merely asserted that it's ridiculous to require such experiments because... miracle drug!

[Dylan16807]

> Again, this is the very thing that needs to be shown. You're simply repeating the same mistake.

"This" being the idea that "compelling evidence" and "evidence from randomized controlled trials" are not the same thing? That has been shown. The cliche example is parachutes. We have compelling evidence that they save lives, even though nobody has ever done a proper trial.

Experiments don't have to be perfect to provide valid data. And there are things you can learn from observation even without an explicit experiment.

Nobody is suggesting that we would take an unknown drug and call it a miracle drug without evidence. When someone has a miracle drug in a hypothetical, pretend they're holding a bottle of penicillin and they've gone to a country where nobody has ever heard of penicillin. How should that country react when it immediately cures almost everyone with a certain disease? Do they really need a randomized trial to be confident it works?

[h_r]

I hardly know where to start with this. Parachutes are not a cliche example of anything other than engineered products that are the result of well understood principles of physics, knowledge which was hardly arrived at by magical intuition. And medicine lags far far behind in the maturity of the science (my understanding as a layman).

And your final example of showing up in scientifically illiterate society wielding a miracle drug and declaring that they need no evidence of its efficacy, ignoring the fact that such evidence was all on you prior to arriving there? Sorry, I don't see how this is productive.

[Dylan16807]

Parachutes are a device for saving lives.

We know they work. We have evidence of efficacy. But this evidence does not come from randomized controlled trials.

If it was a pill, the standards for evidence should not change.

Randomized controlled trials make things easier to prove. But they are not the only way to collect evidence of efficacy.

> And your final example of showing up in scientifically illiterate society wielding a miracle drug and declaring that they need no evidence of its efficacy, ignoring the fact that such evidence was all on you prior to arriving there?

No evidence?? That's the exact opposite of what I'm trying to say.

Your objections... okay, would a better thought experiment be an invention of a new antibiotic?

There is no existing proof of anything.

I go to the nearest town and give the pill to every single resident with gonorrhea, 50 of them, and a few days later only 48 of them have the disease any more.

They otherwise took no drugs they hadn't already been taking for months.

I didn't randomize at all, and I didn't set up a control group.

But it's statistically impossible for that to happen by chance.

Also, I and a team of respected researchers searched for confounding factors just as hard as someone performing a randomized controlled trial, and we could not find any.

Did I provide evidence of efficacy? If not, why not?

[stewbrew]

Unless you did such a trial, you cannot be sure it actually was the drug that helped. But if everyone gets cured, such a trial should be easy to set up. Unfortunately, things are not that easy in real life. There is no miracle drug.

[hexl]

But you already have a control which is a statistic called "survival rate after x years". If you beat it with a large enough treatment group, then you can prove it works.

[stewbrew]

Selection bias?

If you don't control who gets the treatment, who gets a placebo, and who gets the current state of the art treatment, you cannot really exclude your results are biased. There are some ideas about how to reason with observational data but your conclusion are still weaker as with experimental data.

[wolco]

Yes because some also explode after treatmenr. Side effects can be serious.