[aashiq]

It depends! There should probably also be a process by which reviewers themselves get graded. Then paper writers can choose whether to splurge for fewer really amazing reviewers, or a larger quantity of mediocre reviewers. Also, readers will be able to see the quality of the reviewers that looked at a preprint.

[BlueTemplar]

How do you have all three of anonymous authors, anonymous reviewers, and reviewer ratings ?

[bumby]

It might be possible to have a third-party manage the reviewer ratings. Although I suspect some fields are so small/niche that if someone wanted to associate some random ID with a real person, they could match writing styles etc.

[BlueTemplar]

How isn't that third party just reinventing journals ?

[bumby]

The product is different. Consumer Reports isn't reinventing cars; it provides a service by independently rating cars.

[bumby]

Splurging for "amazing reviewers" could also be gamed to "splurge on those reviewers who are likely to rubber-stamp my submission to get paid" (not unlike some of the questionable open-access journals' current business models).