[LetThereBeLight]

More specifically, this paper is focused on the social sciences. That's not to say that this isn't present in the basic sciences either.

But one other thing to note here is that these headlines about a "replication crisis" seems to imply that this is a new phenomenon. Let's not forget the history of the electron charge. As Feynman said:

"We have learned a lot from experience about how to handle some of the ways we fool ourselves. One example: Millikan measured the charge on an electron by an experiment with falling oil drops, and got an answer which we now know not to be quite right. It's a little bit off because he had the incorrect value for the viscosity of air. It's interesting to look at the history of measurements of the charge of an electron, after Millikan. If you plot them as a function of time, you find that one is a little bit bigger than Millikan's, and the next one's a little bit bigger than that, and the next one's a little bit bigger than that, until finally they settle down to a number which is higher. Why didn't they discover the new number was higher right away? It's a thing that scientists are ashamed of—this history—because it's apparent that people did things like this: When they got a number that was too high above Millikan's, they thought something must be wrong—and they would look for and find a reason why something might be wrong. When they got a number close to Millikan's value they didn't look so hard. And so they eliminated the numbers that were too far off, and did other things like that ..."

https://en.wikipedia.org/wiki/Oil_drop_experiment#Millikan.2...

[analog31]

Something that I think the physical sciences benefit from is the ability to look at a problem from more than one angle. For instance, the stuff that we think is the most important, such as the most general laws, is supported by many different kinds of measurements, plus the parallel investigations of theoreticians. A few scattered experiments could bite the dust, like unplugging one node in a mesh network, and it could either be ignored or repaired.

The social sciences face the problem of not having so many different possible angles, such as quantitative theories or even a clear idea of what is being tested. Much of the research is engaged in the collection of isolated factoids. Hopefully something like a quantitative theory will emerge, that allows these results to be connected together like a mesh network, but no new science gets there right away.

The other thing is, to be fair, social sciences have to deal with noisy data, and with ethics. There were things I could do to atoms in my experiments, such as deprive them of air and smash them to bits, that would not pass ethical review if performed on humans. ;-)

[dron57]

Your example of looking at a problem from more than one angle made me think of the problem of finding the Hubble constant that describes the rate of expansion of the universe. There are two recent methods which have different estimates for this rate of expansion.

PBS Space time has an excellent video on the topic: https://www.youtube.com/watch?v=72cM_E6bsOs

[analog31]

Indeed, one of the things that's possible in physics is to nail down the experimental results to the point where, if two results disagree, you know that they really disagree, and that it's not just a statistical fluke. Then it gets interesting.

In physics, when a result raises more questions than it answers, we call it "job security." ;-)

[pavon]

> More specifically, this paper is focused on the social sciences.

No, it isn't. It looked at a few different fields, and found that the problem was actually worse for general science papers published in Nature/Science, where non-reproducible papers were cited 300 times more often as reproducible ones.

[nebulous1]

I think you might be mistaken. The study of Nature/Science papers was "Evaluating replicability of social science experiments published in Nature and Science between 2010 and 2015"

[lupire]

Feynman's examples is of people being more critical about certain issues. A better example is the case of "radiation" that could only be seen in a dark room in the corner of your eye, which turned out to be a human visual artifact and wishful thinking.

[perl4ever]

I assume you're thinking of N-Rays and Blondlot, but there is another phenomenon that more or less fits your description.

https://en.wikipedia.org/wiki/Cosmic_ray_visual_phenomena

It's interesting that according to the Wikipedia article it's not entirely certain whether the radiation is producing actual light or just the sensation of light.