[fchopin]

> and get some sort of crowd-sourced consensus from researchers with credible reputations.

Would be nice, but where would the money come from? Even if you live in a country with high taxes that fund ridiculous projects, they are still often accountable to the public. Would the public want to fund new research or try to replicate or disprove other studies? The only way to make it work would be to build-in a certain percentage of funding for such a purpose in reaction to some gross oversight, so you could get the public behind it, or perhaps get some wealthy donors behind the cause, but I think you'll have a difficult time.

The better option might be having research students be required to replicate at least one study that hasn't been replicated in addition to doing their own research. That way you get some free labor!

[apathy]

Depends. I would like to think that "the public" would notice all those silly cancer patients showing up to clinic (don't they know that cancer is cured twice a week at every major hype factory, err, university?) but, no.

Pie in the sky it is, then! If only a field existed that attempted to quantify the expected variation from a given size and design of experiment.

That field would be of great use to major nation states. So much so, you might call it statistics. The practitioners would probably be big party poopers, always referring to weird concepts like "regression to the mean", "power", "effect size" and other unexciting yet critical details. I imagine there would be funny videos on YouTube about these people, and "real" researchers would happily ignore them while they jerk each other off in P01 study sections.

Anyways, let's keep science the way it is. There's no way the process or its results could possibly be improved!

[fchopin]

> Anyways, let's keep science the way it is. There's no way the process or its results could possibly be improved!

Let it be know that this is sarcasm, for those that have trouble determining that.

Note: I was being completely serious about having graduate students being required to replicate studies that have not been replicated, or to disprove them. That was not sarcasm.

[apathy]

Graduate students are effectively required to replicate studies already. They just don't get any credit for the work when it turns out that the flashy paper didn't replicate, and the journal editors hate to look foolish, so it's exceedingly rare for failures to replicate to see a journal. So the next graduate student to work on the technique also gets to waste months or years on it.

Really a splendid system, isn't it? Sometimes (rarely) I sympathize with industry types who want grant claw-back processes. Then I remember that those are the same crooks that pushed Vioxx and 510k "equivalent" medical devices. And the sympathy evaporates, because they're even worse.