[snowwrestler]

> Part of the problem is we got tricked into thinking "peer reviewed" meant "true," or at least something like it.

No actual working scientist thinks this.

“Glitchc” has it right elsewhere in this thread: the motivating force behind journals is prominence and reputation, not truth.

[D-Machine]

Ah, but the naive public still broadly believes in peer review, and that high profile journals do good review. And the prominence and reputation that comes from these journals arguably then relies on this (increasingly false) public perception.

Would scientists feel the same if the public was more educated about how bad journals and peer review are? Not so easy to disentangle IMO.

[snowwrestler]

The naive public does not believe anything in particular about peer review. They think new scientific results are significant when they read about them in the popular media, that’s it.

People who do need to work professionally with peer review, do understand what it actually does and its limitations.

You seem stuck somewhere in the middle, caring deeply about a system you don’t seem to fully understand.

[D-Machine]

> The naive public does not believe anything in particular about peer review

You'd need to provide evidence or an argument for this. The media reports on things in part based on journal prestige, and likely when questioned, people will say they can trust such things because good scientists have looked at the work and say it is good. This would be an implicit belief that peer review is generally working well, even if they don't use the term "peer review".

> You seem stuck somewhere in the middle, caring deeply about a system you don’t seem to fully understand.

Extremely presumptuous, as I work in this system, and have provided plenty of evidence for my claims. You've provided only sneers.

[snowwrestler]

You've provided evidence that prominent journals experience retractions, fraudulent results, etc. All true. But it is not the job of peer reviewers to decide what gets published.

You've provided evidence that peer-reviewed science often turns out to be incomplete, inaccurate, wrong, fraudulent etc. All true. But it is not the job of peer reviewers to assure completeness, accuracy, or freedom from fraud.

A peer reviewer reads a paper and make comments on it. That's it! They don't check primary data, they don't investigate methods, they don't interrogate scientists, they don't re-run experiments just to double check. They assist a journal's editors in editing--that's it.

The check on published scientific results is the scientific process itself, not the publishing process. Prominent results attract further investigation, which confirms or disproves the reality of the underlying phenomena. Again: that's not the job of peer review.

Do some people ascribe too much authority to peer review? Yes, for sure. IMO your comments in this thread are exacerbating that problem, not addressing it.

[D-Machine]

> A peer reviewer reads a paper and make comments on it. That's it! They don't check primary data, they don't investigate methods, they don't interrogate scientists, they don't re-run experiments just to double check. They assist a journal's editors in editing--that's it.

Um, what? I have done all these things in reviews, and know other academics that have done these things as well. More confusingly though, if you are saying most reviewers don't do these things (which I agree with), this would only strengthen my point?

I'll let readers decide if it is my comments that exacerbate the problem, or if, perhaps, it is apologism for journalistic peer review that might be causing bigger issues in the present day.

[snowwrestler]

Would be interesting if you would be willing to share a paper you reviewed and detail your review process of it. I don't see how one could check primary data or interrogate scientists in a blind review process, for example.