[dmoy]

It is a non trivial problem to do just that.

It's related to the same problems you have with e.g. Sybil attacks: https://en.wikipedia.org/wiki/Sybil_attack

I'm not saying it wouldn't be worthwhile to try, just that I expect there to be a lot of very difficult problems to solve there.

[yorwba]

Sybil attacks are a problem when you care about global properties of permissionless networks. If you only care about local properties in a subnetwork where you hand-pick the nodes, the problem goes away. I.e. you can't use such a scheme to find the best paper in the whole world, but you can use it to rank papers in a small subdiscipline where you personally recognize most of the important authors.

[phi-go]

With peer review you do not even have a choice as to which reviewers to trust as it is all homogenized by acceptance or not. This is marginally better if reviews are published.

That is to say I also think it would be worthwhile to try.