[jpdoctor]

I think the blog post missed the most important point of the NYT piece: Too often, he says, trials are against “a straw-man comparator” like a placebo rather than a competing drug. So the studies don’t really help us understand which treatments for a disease work best.

The standard is usually a placebo. This is done when you are not even sure the drug will have an effect.

The standard should be a competing drugs, otherwise you have no measure of whether the drug under trial is better than current drugs.

The financial incentives are not present for any of the parties (including the regulators) to use the higher bar.

[potatolicious]

I wonder if this will have unintended economic consequences. Clinical trials are astonishingly expensive - if we required all players to clinically test against competitors (or just the current leading competitor), it seems this would create a perverse incentive to be first to market. Not only would you rake in the early profits, but you'll also significant raise the bar for anyone trying to unseat you.

I'm not convinced this will actually be the case, but it's a possible outcome worth thinking about.

One thing I'm wondering: if all clinical studies are quantified via efficacy vs. a fixed placebo, shouldn't that make the results comparable? If Drug A is 200% more effective than a placebo, and Drug B is 300% more effective, does that not suggest that Drug B > Drug A?

[refurb]

Yes, it is possible to compare two placebo controlled trials and get a rough estimate of the relative efficacy of the two drugs. Many of the prescribing decisions that physicians make are based on these indirect comparisons.

However, it's only a rough estimate. Even if the patient populations are nearly identical, you can often see different outcomes.

To give you an example: Crohn's disease is an autoimmune disease of the large (and sometimes small) intestine. If you look at the clinical trials for the biologics used to treat the condition, you'll see remarkably varied outcomes, even in the placebo arm.

In other words, if drug A show 50% vs. 20% efficacy vs. placebo and drug B showed 70% vs. 40% efficacy vs. placebo, which is the more efficacious drug?

[roc]

> "drug A show 50% vs. 20% efficacy vs. placebo and drug B showed 70% vs. 40% efficacy vs. placebo"

Which raises a further (or more basic) confounding factor: the placebo effect isn't fixed.

It does vary between trials and even appears to be steadily increasing in potency over time. [1]

So first-to-market drugs have an added advantage when naively considering "improvement vs placebo" -- as their test were run years ago, when the placebo effect itself was a weaker opponent.

[1] http://www.wired.com/medtech/drugs/magazine/17-09/ff_placebo...

[pdpi]

> One thing I'm wondering: if all clinical studies are quantified via efficacy vs. a fixed placebo, shouldn't that make the results comparable? If Drug A is 200% more effective than a placebo, and Drug B is 300% more effective, does that not suggest that Drug B > Drug A?

I'm not sure that works. How do you account for different test conditions?

[shoyer]

Exactly -- the tests may have been done on different populations. In an extreme case, imagine both treatments are randomized trials for an otherwise terminal illness, but Drug A was tested on the elderly and Drug B was tested on youths. When cured, those on Drug B will live longer, if only because they are young.

[shoyer]

It may be "too often" that new treatments are compared to placebos, but it's not the prevailing practice, at least for serious indications like cancer. It would clearly be unethical to deny people proven treatments would extend their lives. To quote cancer.gov (http://www.cancer.gov/cancertopics/factsheet/clinicaltrials/...):

  10. Are placebos used in cancer treatment clinical trials?

  The use of placebos as comparison or “control”
  interventions in cancer treatment trials is rare. If a
  placebo is used by itself, it is because no standard
  treatment exists. In this case, a trial would compare the
  effects of a new treatment with the effects of a placebo.
  More often, however, placebos are given along with a
  standard treatment. For example, a trial might compare the
  effects of a standard treatment plus a new treatment with
  the effects of the same standard treatment plus a placebo.

[cowsandmilk]

I really would like to see this claim from the article verified. Most clinical trials I'm aware of are not truly against placebo, especially not cancer trials. Even the Avastin trial mentioned in the article is not truly against placebo.

The two arms were: (a) standard treatment + placebo (b) standard treatment + Avastin

So, while it is true that placebo was compared to Avastin, it could be that Avastin acts identically to current standard of care. That is, Avastin could show no impact in this trial and show impact in "just Avastin" vs. "just placebo".

In most progressing fatal diseases, it is unethical not to provide standard of care, so trials are set up such that all patients receive at minimum current standard of care.

[Retric]

Forcing a comparison to existing drugs sounds good, but things that are less effective but have fewer side effects are often far more useful than the most potent option aka Tylonol vs Morphine. The problem is you still need the placebo baseline or saline would pass as the safe but less effective alternative. Now, adding competing drugs to a trial is great information the problem is they are already really expencive so adding even more cost is generally a hard sell.

[tehwalrus]

The thing that freaks me out about placebos is that you can rig them - a sugar pill, for example, isn't necessarily inert in diseases that mess with sugar levels (among other things) - by pitting your drug against a "placebo" that actually makes the disease worse chemically, you can get away with murder when your drug magically turns out to be better than it.

Comparing against other drugs, especially established ones, is a reasonable fix for this, although as others have pointed out this may cause perverse incentives.

[ceejayoz]

I'd suspect that diabetes pill placebos would be something like a gelcap filled with water, not a sugar pill.

[mikeash]

Is it not possible to take the competing drug's trial against a placebo, the new drug's trial against a placebo, then compute the new drug's effectiveness compared to the competing drug? Is it just that it's too inaccurate to go through the indirection like that?