[glial]

Replicating studies seems like the perfect training exercise for graduate students. I wish it were a mandatory part of training at most schools.

[yhoneycomb]

The problem is... if you can't replicate the experiment, how do you know if it's the graduate student's fault or the experiments fault?

[rflrob]

I always thought that if I ran my own lab, I would have rotation projects be to replicate a recent result from the lab. Then, a lot of the infrastructure and support would still be there, so it would be pretty clear if the fault lay with the experiment. Plus, it would reinforce the notion among trainees that the point of science is to be replicated, and that the hard part of doing something novel is figuring out what to do, not actually doing the work.

[musicale]

> how do you know if it's the graduate student's fault or the experiment's fault?

Usually by

- checking the original publications to make sure you are repeating the experiment correctly

- checking (and monitoring) the experimental setup to make sure it is doing what you think it is doing and you aren't introducing errors

- checking the data analysis

- running "sanity check" experiments with the same setup to make sure it has no obvious flaws

- comparing with recent replication experiments by other researchers

- showing that the experiment and results are repeatable by multiple people in different organizations with their own lab setups

- consulting with the original authors (who may be helpful or unhelpful) if they are available

- comparing against other data sets

- comparing against results from analytical modeling or simulation

- looking for alternate explanations of the anomalous results and checking for whether they might be occurring

etc.

As others have noted, the same methods apply to sorting out conflicting experimental results regardless of who conducts the experiments.

[glial]

Publish methods and results in a database. Every result will be a draw from a distribution. Today only exciting ones get published but it would be better to see the full distribution.

[adamnemecek]

If it has been replicated many times then you know whose fault it is.

[blackbear_]

The more people fail, the more likely that the fault is on the experimenters' side

[leetcrew]

you could ask the same question of someone with a phd.

[netizen-9748]

This is a very good point, we are all still fallible humans no matter what degree w hold

[QuesnayJr]

Isn't this common, as a warm-up for grad students? It's just hard to publish.

[glial]

Not in my experience, unfortunately.