[Barrin92]

Honestly what we need to do is establish much stronger credentialing schemes. The "only a good guy with an AI can stop a bad guy with an AI" approach of trying to filter out bad content is just a hopeless arms race and unproductive.

In a sense we need to go back two steps and websites need to be much stronger curators of knowledge again, and we need some reliable ways to sign and attribute real authorship to publications. So that when someone publishes a fake paper there is always a human being who signed it and can be held accountable. There's a practically unlimited number of automated systems, but only a limited number of people trying to benefit from it.

In the same way https went from being rare to being the norm because the assumption that things are default-authentic doesn't hold, the same just needs to happen to publishing. If you have a functioning reputation system and you can put on a price on fake information 99% of it is dis-incentivized.

[tbrownaw]

Is this not already a thing? You can look up purported papers by DOI, and whatever journal it came from supposedly had it reviewed and should know who sent it to them.

(And if that doesn't work, how is what you're suggesting meaningfully different?)

[Barrin92]

It's not at all a thing. Here's a recent study looking at citation fraud on Google Scholar including professional citation boosting services including with fake identities. It's widespread practice. https://arxiv.org/abs/2402.04607

Having a machine verifiable, cryptographic identity system that renders these kinds of things transparent, basically the equivalent of a ledger but instead of using it for get-rich schemes using it for identity would probably make verification enforceable.