Hacker News new | ask | show | jobs
by mattkrause 2355 days ago
I’m not saying you shouldn’t be skeptical. The reviewers’ job is to ask whether the approach used and data collected make sense and if so, whether and how well they support the authors’ claims.

At the same time, peer review is meant to be an advisory process, not an adversarial one. You should certainly flag things that seem problematic, and ideally offer solutions, but you’re not expected to—-and cannot, really—-tear down the experiment and root out every possible mistake or malfeasance. All you’ve got is a day or so and a 6000 word description of the project, so if the manuscript claims that the mouse weighed 12.3 grams, you’ve basically got to take that number at face value. Trust might be too strong of a word—-you could certainly question a weight of 123 grams, which is improbably large—-but you should at least start in equipoise with respect to some stuff.

I also don’t see the mutual benefit, beyond nebulous things like enjoying consensus. A short review is also easier to write than a long one, but it’s just as easy to be dismissive as uncritical. Reviews are usually unsigned—-and the authors and reviewers may not even know each other[0] so it’s hard for there to be an explicit quid pro quo.

[0] Of the stuff I reviewed in 2019, I knew exactly one of the authors, and here ‘knew’ means ‘Once amiably chatted with them in a coffee line at a conference’.

1 comments

Asking “does this make sense” when it’s monkey trained to use joystick is something very different than trying to question and confirm “our model which is entirely based on these 40 other models which were all results of estimations of samples taken over the last 100 years then extrapolated out”... you can ask to see the monkey using the joystick.

The model verification would be years of work - so everyone just “trusts” it’s right, although we basically know it’s not because the chance it isn’t is just much more likely.

It doesn’t have to be a quid pro quo any more than it has to be “ugh, that sounds really boring to confirm, I’ll just believe it”.