[smabie]

We should have less transparency, not more. Right now the peer review process is single-blind, but it really should be double-blind.

Even if someone doesn't have a PhD or work at a research institution, they should be able to publish good science. Right now, that just isn't possible. And the opposite problem is also true: if you're a big shot in your field, you'll be able to find at least one journal that will publish whatever crap you submitted, regardless of the quality.

[derefr]

Getting that benefit only requires double-blinding during the review process. There’s no reason that both sides of the blinding cannot be removed (and also revealed to third parties) after the review process is complete.

[smabie]

I can't think of any real upside of that, but can think of a lot of downside. Humans, and especially academics (I've noticed a trend in my work environments: the less money everyone makes, the more power is sublimated into inane dick measuring contests) are petty, and less personal it is, the better.

[derefr]

How about if the anonymity is "de-blinded" only partially, replacing it with pseudonymity?

Imagine a system like this:

1. The reviewers on your paper not only have to collaborate on a decision to accept/reject, but also write an opinion (like a https://en.wikipedia.org/wiki/Legal_opinion) about your paper, individual to themselves, after the consensus to accept/reject is reached (so some of them will likely be https://en.wikipedia.org/wiki/Dissenting_opinion s.)

2. The reviewers are each assigned a global permanent pseudonymous identifier—a UUID, basically—known only to them and some "Society for the Advancement of the Scientific Process in Academia" organization.

3. Every vote a peer-reviewer makes, and also every opinion they write about a paper, must be registered with the same academic-process org, whose job is then to collate and publish them to the Internet under the reviewer's pseudonymous identifier.

You'd be able to use such a website to both 1. audit the peer-review process for a given paper; and 2. cross-reference a given peer-reviewer's votes/opinions.

Additionally, the standards body itself could use the cross-referencing ability to normalize peer-reviewer votes, ala how the Netflix Prize recommendation systems normalized votes by a person's interpretation of the star ratings. (They'd have to ask peer-reviewers to vote with something more fine-grained than a binary pass/fail, but that'd be an easy change.)

The only thing I would worry about in such a system, is that academics might not want the negative opinions of the peer-reviewers on their paper to pop up when random other people plug the paper's DOI into Google Scholar, because a dissent on an accepted paper might unduly impact the paper's impact-factor.

[adminprof]

You can't think of any reason? What about retaliation for being the less enthusiastic reviewer? What about the opposite, that you become known for being an easy reviewer who missed some obvious flaws?

[derefr]

You're still assuming that this happens right away. How about if the blinding is removed after 50 years? Then people studying the history of science would have the data, but it would have no impact on the careers of the people involved.

[Beldin]

Double blind reviews are the standard in at least some corners of CS. Not only conferences, also for some journals.

Though you will also encounter single blind reviewing. I haven't encountered a truly open review process yet.