[2cynykyl]

I have come to realize that when we do a peer review for Journal X, all we are really doing is protecting the journal's reputation by helping weed out substandard papers.

There seems to be a belief that the peer review process is like a universal system, so if a paper is 'rejected' during peer review for Journal X then it will be thrown in the garbage and never trouble us again. But in reality, the authors will submit it to as many journals as necessary until it sneaks through, and eventually it will sneak through.

I repeat: Every rejected paper will eventually be published somewhere. Perhaps not in Nature, or Top Journal A, but it will appear in Respectable Journal B or Mediocre Journal C. And it will bare the seal of "peer reviewed".

Ironically, I guess my point is that peer review is not broken; it's working great for the journals.

[slowmovintarget]

I've found it interesting to hear about the history of peer review, which started as a way to keep out "the unqualified masses" from the review process, and also as a way to sell more journals. The really interesting part of this is that "peer review" wasn't a formal thing before the 1960s. This means that the lion's share of what we consider the greatest advances in scientific endeavor never went through "peer review" in the sense it's meant now. Not Newtonian mechanics, not Maxwell's electromagnetism, not Einstein's relativity, not quantum mechanics... Peer review became about segmenting and exploding the number of journals universities had to purchase to be "current." It was really a publishing company coup on the scientific process.

Eric Weinstein talks about this in his critique of the whole Terrence Howard kerfuffle.

[gus_massa]

> I repeat: Every rejected paper will eventually be published somewhere. Perhaps not in Nature, or Top Journal A, but it will appear in Respectable Journal B or Mediocre Journal C. And it will bare the seal of "peer reviewed".

I agree, but I dont know if it'a a bug or a feature. We had one paper that took a while to get published.

I prefer to use "peer reviewed and published in a good journal" where I'm not sure if "good" means A, B or C in your scale. But there are also D, E, F, G, ... depending on how finely you want to clasify the crap and predatory journals.

The problem is that outside my area, I really don't know which journals are good and which journals are bad. One of my criteria to not send the journal to the Z slot is if I recognize the publisher of the journal. Sometimes the impact index is useful, but it changes too much from area to area. I take a look at the other papers in the same jorunal. Too many single author papers or too many unrelated topics are red flags.

At the end of the day, for research or for posting angry comments in HN, the only solution is to RTFA. I read the abstract, skip the introduction that is usually too optimisitc, and go directlry to the tables and graphics. I learned that Ctrl+F "exclusions" is important in medicine, becuase I've seen weird methods to filter the patients.

[CuriousCosmic]

Honestly I think it's a good system at it's core, it just needs some adjustment on incentives (mainly that reviewers need to actually be fairly compensated for their time).

The way I see it, journals are more or less independent reviewing orgs. That a paper has been peer-reviewed means nothing. It's all in the reputation of the reviewer and what they said.

Personally I'd actually like to see journals publish their reviews including rejections because I think it's way more important to know why something was or wasn't accepted than that it was.

[2cynykyl]

Publishing a record of rejections and reasons is a very interesting idea! I will be adding that to my "deserves more thought" pile!