[Fomite]

Medical and public health researchers are bound to ethical guidelines that would prevent something like this, because the preponderance of evidence is that having health insurance is a net positive for someone's health - the only reason it made sense in Oregon is the fact that they needed a lottery anyway.

As for the Oregon study, the results of that study are still relatively new (the idea that any measure focused on preventative health will show results after two years is pretty suspect). The authors of the study discuss this for diabetes:

"Medicaid significantly increased the probability of being diagnosed with diabetes after the lottery (by 3.8 percentage points, relative to a base rate of 1.1) and use of diabetes medication (by 5.4 percentage points, relative to a base rate of 6.4). As discussed in the paper, based on clinical trial evidence on diabetes medication, we would expect this increase in the use of medication for diabetes to decrease the average glycated hemoglobin level in the study population by 0.05 percentage points, which is well within our 95% confidence interval for the impact of Medicaid on the level of glycated hemoglobin."

[yummyfajitas]

By this logic, it would be unethical for the FDA to demand a random trial for any drug that has some correlation studies showing it is effective.

As for the number you are cherrypicking, it is true that health insurance increased medical consumption (including ER visits, in spite of what ACA supports claimed) among people who received it. However, no measurable effect on health (besides depression) was observed.

[Fomite]

You'll find that clinical trials are regularly halted when the treatment is found to be markedly superior or inferior to placebo or the control group.

The original study that showed aspirin's effect on heart attack prevention comes to mind for the former circumstance, and a number of HIV prevention studies for the latter.

[yummyfajitas]

Halting a clinical trial when the result is clear is very different from skipping the clinical trial on the basis of a correlation study or two. Your comparison is so nonsensical that either you are being deliberately disingenuous or you don't understand statistics at all.

Either way, no point in continuing this.

[Fomite]

I was being charitable to your example, because generally speaking there are no observational studies done before a drug is put up for approval - that's not how the approval pipeline works. The closest I can come up to your example is the occasional off-label use of a drug for some other condition, but the reports from those are largely small n studies. That's entirely different from a series of studies based on NHANES.

Beyond that, in an intentional randomized trial, rather than the 'happy accident' like the Oregon study, the actual control is not 'Nothing' but the medically indicated standard of care. Studies are often required to provide medical care, education, etc. to their participants. I cannot imagine a study managing to get "We deny a bunch of folks health insurance" by an IRB unless it was an externally forced process, like the Oregon study.

Your insult about not understanding statistics, in addition to being off-base, is rather spurious. This isn't a statistical question, it's a public health ethics question. Statistics doesn't really come into whether or not "Keep a bunch of people from accessing healthcare" will get nailed by an approval board.

Also, the FDA often does take observational evidence into account, especially when expanding things like what age range a drug is medically indicated for.

[Fomite]

While we're on the topic, Bayesianwitch.com's description:

"Your new homepage goes viral, but you aren't sure what copy is converting. Hook that copy up to BayesianWitch and only converting copy will be showing. No waiting days for the answers from an A/B test."

Seems to imply a reliance on "correlational" data. Is there some hidden randomization in there? Or are you giving your clients a lower standard of evidence?

[yummyfajitas]

We are using a lower standard of evidence than medical decisions. The goal is to get as many clicks/conversions as possible in aggregate, most of the time. I.e., if you have a call to action ("3 day sale", "spring sale", "march sale") that dies in 3 days, we'll do the best we can to increase your conversions in those 3 days.

Similarly, if you have 10,000 seo-optimized microsites, each with traffic too low for a per-site A/B test, we'll improve your conversion rate across the 10,000 microsites.

If you want to make a long term change (e.g., logo, button color, feature) for a high traffic site (your one and only landing page) you are better off using a traditional A/B test.