[scarmig]

Millikan's oil drop experiment on quantization of charge and measurement of e is an infamous example. No one can reproduce his reported results and, in some ways, his reported results go beyond what's attributable to bias and cross the line into fraud. And yet his ultimate result was true enough, and he progressed science enough to win a Nobel.

[fastball]

Millikan's oil drop experiment is definitely reproducible.

He excluded values to reduce his error (seems like), but that doesn't make the experiment non-reproducible. You can do the same experiment today and get an approximate value for e.

Will you get the exact same results as Millikan? Of course not. But that's not what reproducible means.

[scarmig]

The issue is that you get substantively different results from Millikan if you follow his stated experimental procedure. The only way to reproduce his results is to perform the experiment and then selectively remove datapoints that disagree with his result until you only have results remaining that are close to his. That doesn't qualify as reproducible in my book, except in the trivial sense you can perform the same experiment he did, irrespective of the actual result.

[fastball]

The goal of the experiment is not to get a certain arbitrary number (the one Millikan came up with), the goal is to get the approximate value of the elementary charge.

Reproducible means that if you follow the method outlined by Millikan, you should get an approximate value for e. This is indeed the case, so the experiment is reproducible.

A non-reproducible experiment would be one where you follow Millikan's method but get something that is nowhere close to being a value for e.

[scarmig]

You're using a n̶o̶n̶s̶t̶a̶n̶d̶a̶r̶d̶ different sense of the term reproducible. Just to drive the point home, it wouldn't make sense to talk about a reproducibility crisis if having data being consistent across different experiments conducted by the same methodology with different researchers wasn't a key component of reproducibility.

ETA: I encourage the interested to read https://plato.stanford.edu/entries/scientific-reproducibilit... which delves into these semantic issues.

[fastball]

Depends on the experiment. Oil drop is based around something we "know" exists (the charge of an electron) and is just trying to measure it. Measuring a fundamental value is a very different kind of experiment than trying to determine if something exists at all in a wildly multi-variate system (most of the research suffering from reproducibility issues).

Not every experiment needs to be reproducible in exactly the same way, just like not every study needs to be double-blind. You need to interrogate the reason the experiment exists in the first place.

I guess another way of saying it is this: even though Millikan's actual final result is not reproducible because he fudged his numbers, the scientific method employed in the experiment is valid, which is really all you need in the case of that experiment.

The problem with medicine / psychology / other less "hard" research is that in many instances it doesn't matter if the method is reproducible if the results are not. If the goal is to show that eating pancakes cause depression the result is actually all that matters.

Thanks for the link, I will give it a read.

[throwaway2245]

> I encourage the interested to read https://plato.stanford.edu/entries/scientific-reproducibilit...

This link clearly talks about reproducible results, not reproducible data.

Wherever there is some degree of randomness, we would not expect to get the same data, but we would expect to draw the same conclusion.

See also: the Mendelian paradox - it is sometimes argued that Gregor Mendel's results were too perfect, and yet he is honoured as the father of genetics.

[dekhn]

as several of my genetics teachers put it: "wow, mendel was really so lucky that he just randomly managed to pick 7 independently associating traits on different chromosomes" (the molecular biologists in the audience all shook their heads)

[bigbubba]

Physics classrooms around the world have had students replicating this experiment for decades. I think it's not his definition of 'reproducible' that's nonstandard.

[dekhn]

science got the right place eventually. It was reproducible enough that the field made forward progress and even did extraordinary analysis to find out the originator showed some judicious data reduction.

[edjrage]

So it worked despite all the bias and fraud. You could even call it luck. The reproducibility implied by the fact that at least some of it is true is not a "nice-to-have", it's the whole point of it working. We wouldn't call it true if we couldn't confirm it to be true.

To me it seems you're conflating reproducibility with insight. Of course insight is also key, and it may be useful even if it's wrong, as long as it causes others to recursively have new insights and create hypotheses and do experiments that eventually turn out to be true.

[scarmig]

I would say that Millikan had a fundamentally true insight, and he published it to the world with an experiment whose results weren't reproducible.

[edjrage]

Exactly. Which renders the logic behind the experiment false. But since the insight was true, it eventually led to experiments that yielded true results. (Notice I didn't say the insight was false, I supported your idea by emphasizing that even false insights can be useful.)