[scarmig]

There's a segment of the population that sees scientists as materialist, secular replacements for priests and mystics, instead of sophisticated monkeys trying their best to make sense of a messy, confusing world.

As a side note, I'd push back against the idea that replicability is the key ingredient of science: it's neither necessary nor sufficient to discover scientifically "true" knowledge, though it's definitely a nice to have. Plenty of important measurements have been made confirming theories we see as true today (or true enough) that turn out to have been irreproducible, plagued by experimenter bias, or even outright fraudulent. But it's a particular research programme, embedded in a social context, that generates knowledge, not a single reproducible experiment.

[pushrax]

My take on science is defined by the scientific method:

- we want to be able to understand and predict things

- we can come up with various explanations that might be able to do this

- if our explanation fails to predict an observation, it is wrong and we need to update or replace it

If you're doing that, you're doing science.

[Complexicate]

This is very close to Richard Feynman's explanation of the scientific method: https://youtu.be/kBqemHR49-c?t=33

" Guess -> Compute Consequences -> Compare to Nature / Experiment / Experience / Observation

If it disagrees with experiment, it's wrong. In that simple statement is the key to science."

[scarmig]

In a universe with spherical cows, maybe. But the issue is that theories we view as true (at least within their scope) often fail to predict observations. Which is fine: we modify them with ad hoc hypotheses to round off the rough edges. The issue is that all theories, true or false, do that. Copernican theories were initially less predictive than Ptolemaic ones, for instance; a simple heuristic of rejecting theories that generate more incorrect predictions than other theories would have left us committed to a Ptolemaic universe. And yet we moved.

[pushrax]

I'm not saying to follow a purely incremental optimization path, leading to local maxima. Thinking outside the box is clearly important for generating new ideas to test. We're not done until we run out of things to perfectly predict.

I would say some of the ancients did science too, to the best of their knowledge.

[visarga]

> But the issue is that theories we view as true (at least within their scope) often fail to predict observations. Which is fine: we modify them with ad hoc hypotheses to round off the rough edges.

This passage reminds me of how neural nets learn and often fail to generalise.

[KKPMW]

> But it's a particular research programme, embedded in a social context, that generates knowledge, not a single reproducible experiment.

Somewhat disagree. That particular research programme must still have reproducible experiment as a unit. A million irreproducible experiments will generate exactly zero knowledge.

[clairity]

you see this sort of (religious) appeal to authority prevalently on topics like mask wearing, with phrases like “based on science” and “science tells us...” to justify largely politically-based beliefs, in many cases citing administrators or doctors (who are generally not first and foremost scientists) as ‘evidence’ (npr & nyt literally does this every day).

science has told us so far that masks might help at the margins, but hasn’t proven it, which indicates it probably shouldn’t be counted on as a primary mitigative measure, but the public signaling value is just too irresistible for the believers (transmission principally happens in private where social norms oppose mitigative measures).

[BlueTemplar]

Doctors' hands on experience gives them a special kind of knowledge pretty close to scientific. Also, science doesn't really proves, but rather disproves, and the "softer" the science, the harder it is to get any certainties... (And medicine, unlike biology, is a soft science.)

[clairity]

it's not that doctors don't have specialized knowledge that might be stochastically predictive in specific cases, but that their experience, especially around something like a pandemic, skews their perspecive in ways that are very difficult to self-identify and compensate for. and that's on top of doctors holding their own idiosyncratic sociopolitical views too.

in other words, expertise is narrow, and treatment experience isn't research.

[BlueTemplar]

Everyone has their own idiosyncratic sociopolitical views. I would not discount doctors' specialized knowledge as 'not science' (or worse, 'not evidence'). You just have to remember that medecine is a mix of hard and soft sciences (and other things too).

[clairity]

that's intuition, not science. that doesn't mean it's not useful, it's just not useful as science. the point of the appeal to authority fallacy is to identify and unbias us against that kind of rhetorical trap in argumentation.

[marketingPro]

I don't understand this. If I spend my life designing bridges I still have to prove it with math. What kind of medicine is not science but "hands on"?

[BlueTemplar]

Bridges are a good example. You have to remember that bridge building (as pretty much any field of engineering or even science) didn't start out as math-first, but as experience-first. And there's really no opposition there, after all whole fields of science (the 'soft' sciences) don't (can't) even use math ! Medecine is a mix of hard and soft sciences (and other things too).

[fastball]

Maybe that is because there is a segment of "scientists" (read: people who publish in "scientific" journals) that are just materialist, secular replacements for priests and mystics.

Can you give an example of something that isn't reproducible but was still true?

[scarmig]

Millikan's oil drop experiment on quantization of charge and measurement of e is an infamous example. No one can reproduce his reported results and, in some ways, his reported results go beyond what's attributable to bias and cross the line into fraud. And yet his ultimate result was true enough, and he progressed science enough to win a Nobel.

[fastball]

Millikan's oil drop experiment is definitely reproducible.

He excluded values to reduce his error (seems like), but that doesn't make the experiment non-reproducible. You can do the same experiment today and get an approximate value for e.

Will you get the exact same results as Millikan? Of course not. But that's not what reproducible means.

[scarmig]

The issue is that you get substantively different results from Millikan if you follow his stated experimental procedure. The only way to reproduce his results is to perform the experiment and then selectively remove datapoints that disagree with his result until you only have results remaining that are close to his. That doesn't qualify as reproducible in my book, except in the trivial sense you can perform the same experiment he did, irrespective of the actual result.

[fastball]

The goal of the experiment is not to get a certain arbitrary number (the one Millikan came up with), the goal is to get the approximate value of the elementary charge.

Reproducible means that if you follow the method outlined by Millikan, you should get an approximate value for e. This is indeed the case, so the experiment is reproducible.

A non-reproducible experiment would be one where you follow Millikan's method but get something that is nowhere close to being a value for e.

[scarmig]

You're using a n̶o̶n̶s̶t̶a̶n̶d̶a̶r̶d̶ different sense of the term reproducible. Just to drive the point home, it wouldn't make sense to talk about a reproducibility crisis if having data being consistent across different experiments conducted by the same methodology with different researchers wasn't a key component of reproducibility.

ETA: I encourage the interested to read https://plato.stanford.edu/entries/scientific-reproducibilit... which delves into these semantic issues.

[dekhn]

science got the right place eventually. It was reproducible enough that the field made forward progress and even did extraordinary analysis to find out the originator showed some judicious data reduction.

[edjrage]

So it worked despite all the bias and fraud. You could even call it luck. The reproducibility implied by the fact that at least some of it is true is not a "nice-to-have", it's the whole point of it working. We wouldn't call it true if we couldn't confirm it to be true.

To me it seems you're conflating reproducibility with insight. Of course insight is also key, and it may be useful even if it's wrong, as long as it causes others to recursively have new insights and create hypotheses and do experiments that eventually turn out to be true.

[scarmig]

I would say that Millikan had a fundamentally true insight, and he published it to the world with an experiment whose results weren't reproducible.

[edjrage]

Exactly. Which renders the logic behind the experiment false. But since the insight was true, it eventually led to experiments that yielded true results. (Notice I didn't say the insight was false, I supported your idea by emphasizing that even false insights can be useful.)

[renewiltord]

I imagine the LD50 of many substances are defacto non-reproducible despite being mechanically replicable. i.e. the world will not permit us to test the LD50 of FOOF so any measure of its danger to human lives is defacto untestable. Yet we have a notion of its truth.

Of course there is an epistemological distinction here between a thing that cannot be reproduced by its nature vs. one that cannot be reproduced by societal norms.