[TeMPOraL]

> But what really gets me is the disconnect between "most scientists agree there is a reproducibility crisis" and "most scientists believe most of the papers they read are basically true and reproducible".

It sounds very much like an academic equivalent of Gell-Mann Amnesia - the phenomenon in which you notice that every news article on a topic in your field is complete garbage, people you know in other professions report the same situation with articles on topics in their fields, and yet when you turn the page and see an article on something outside your field, you forget about the whole thing and treat it as a gospel.

[agumonkey]

I find the science crisis even more worrying now that society and Internet spread the notion of "sourced everything or it's false".

if even 'scientist' themselves fail, then society is heading toward an absurdly frightening faux-intellectual inquisitive period.

[SlowRobotAhead]

Agreed! Seen with “Fact checking” websites aren’t about nuance, just a political position that argues “the part of the truth we want you to spread”.

[agumonkey]

I wouldn't conclude that much. I think they just believe that a paper is an absolute truth but most of them don't have enough knowledge about the ways and history of the scientific field. To me it's mostly an anxiety based reaction due to this era lack of 'promises' and emerging problems. People are running for certainty.

[SlowRobotAhead]

I would.

Fact checking, unfortunately, isn’t what we think it is. Despite the superficial appearance, fact checking isn’t a helpful tool for determining the truth and for forming an accurate opinion. Instead, it’s actually an in/out group filter which segregates people by belief and value, while allowing each group to believe they hold the Factual High-Ground, and to claim any subsequent moral position which proceeds from being “factually correct.

[dragonwriter]

> Despite the superficial appearance, fact checking isn’t a helpful tool for determining the truth and for forming an accurate opinion

Sure it is. Not the bare conclusions viewed uncritically, but the support with references, if it exists (which in most notable fact checkers it does) absolutely is.

[inimino]

Isn't it the exact opposite? Presumably the papers scientists are reading are primarily the ones in their own field.

[Frost1x]

That depends on how critical you are. Our current incentive structure is out of control across the board and everything is skewing towards maximizing ROI in about every aspect of life money is involved in.

I apply this perspective to everything I see ("how are they getting money from this?") then work backwards and it seems to lead to accurate predictions, at least anecdotally. Perhaps I'm just jaded and cynical but it works well, unfortunately.

[TeMPOraL]

> Perhaps I'm just jaded and cynical but it works well, unfortunately.

It works well for me too.

I've learned to extend the "how are they getting money from this?" question into a generalized, first-principles thinking - look at the incentive structures and think about what they imply, about what's the expected behavior of a system running under those incentives. Because while not everything is about the money, systems will evolve over time along the lines of the incentives contained in them.

[TeMPOraL]

I'm arguing that the phenomena seem identical in their underlying structures. In both cases, we're dealing with a situation in which a person faces mountains of evidence that a source (a newspaper, or scientific papers in a given field) keeps pumping out inaccurate or wrong publications, but despite all that evidence, they assume that whatever isn't explicitly pointed out as wrong must be 100% true, accurate and honest.

[inimino]

Ah, yes. Wishful thinking.

[asdff]

Ideally, but once you are deep in a field what a field even is becomes murky. You have a biology paper you submit for review, but methodologically it's not really a biology paper, but maybe a statistics paper, or a computer science paper, or a theoretical math paper. Who reviews that paper, the person who knows the biology in question or the methods through and through? Sometimes the only person in the world who knows the theory and the technology in question the best is the author.

[jancsika]

> people you know in other professions report the same situation with articles on topics in their fields

The speculation referred to as the "Gell-Mann Amnesia effect" starts with a subject identifying a low-quality article riddled with errors from a section of the newspaper within their area of expertise. They then turn the page to another section of the newspaper outside their area of expertise and "treat it as gospel" without thinking critically.

Maybe it's true, maybe it's not. But you are adding another part-- colleagues who use relevant expertise to inform the subject that the other sections of the newspaper are also low-quality and riddled with errors.

With that addendum I'm confident that I can now beat the house. I will take the bet against this modified Gell-Mann Amnesia effect for any amount the casino is willing to let me wager.

Edit: clarification