[c0achmcguirk]

This part was most interesting to me:

"Incidentally, when you write a paper, you can expect most reviewers (and readers) to make only one pass over it."

I understand reviewers are busy, but we depend on peer review to filter out bad or poorly-researched material. I don't think one pass is enough.

Obviously, so does the author.

[seanmcdirmid]

Depending on the conference or journal, most papers can be rejected in one pass. Heck, the reviewer might abort after the intro if the paper is particularly bad.

The papers that make it past one pass get more scrutiny.

[UncleMeat]

I've gotten reviews where it was obvious the reviewer didn't read the paper. "You didn't address X at all" when I had a section heading under Discussion labeled X in big bold font.

The review process is stochastic and leans towards reject by default.

[axlprose]

Indeed, but then this is somewhat to be expected when the peer review process itself doesn't have much in the way of incentives for the reviewers. Journals ask reviewers to voluntarily put their own work on hold for a potentially non-trivial period of time to grok a non-stop fire hose of papers, just so their and the original authors' efforts end up buried under a mountain of other research behind some absurdly expensive paywall later. And if you regularly slip up on deadlines or opt not to play along with these requests for some reason or another, good luck maintaining good relations with that journal in the future for your own papers.

Morals and civic duty aside, it's hard to blame reviewers for skimming or 'phoning it in' under such conditions. With all these subtle politics and screwy incentive structures, I'm unfortunately not surprised at all that science is having a replication crisis.

[ISL]

More practically, one should not expect that a peer-reviewed paper is correct.

For a YC-compatible analogy: think of it as code review. Pure junk gets filtered out, but bugs inevitably remain.