[azan_]

People should get paid for reviewing. That’s the solution. Publishing companies rack in billions of dollars in pure profit exploring free labor. Once people actually get paid for reviewing, it becomes much faster and higher quality and you won’t need AI triage.

[Sharlin]

Doesn’t work anymore because people would just have ChatGPT generate their reviews and get paid for doing nothing.

[zx8080]

So in sum:

- it did not work before because greed

- will not work anymore because AI

Is human science dead?

[inigyou]

Science as an activity will never die, but science as an institution is in big trouble.

[NetMageSCW]

That should be will not work anymore because of greed.

[StableAlkyne]

Nah, people will just come up with different (probably broader) heuristics when reviewing.

We have a past example of this in the US, too. Certain countries have historically had a huge problem with paper mills. Because of this, most people in the US/EU do have a negative bias once they see the author affiliations of those countries. Yes, it isn't fair to researchers who aren't pulling some shenaniganry. But it is a common shortcut; the sixth or seventh time you've spent a few hours reviewing a paper filled with bullshit, you probably would develop it too.

I imagine we will see similar things come up: the most obvious I can think of is, if it's not a well-known institution then it might be bullshit.

Hell, we are already seeing something similar in FOSS, where many high profile projects have completely banned AI contributions due to all the low effort slop PRs.

[afdbcreid]

Then perhaps people need to pay to get their paper reviewed?

[azan_]

People already do.

[tollgategit]

Alternatively, put the pressure on the incoming requests for review.

Pay a deposit to have your work reviewed, if it's accepted as a good faith submission, the deposit is returned minus a minimal, irrelevant fee, if it is deemed in bad faith, take all of the deposit as a time-waste tax. With enough cases going wrong, the flood of slop slows down and maintainers have to pull the trigger as often.

This was the solution several people had proposed for Git issues, but none that I know of took the plunge. I really think curl should have done that.

[NetMageSCW]

Isn’t that an incentive for the AI under every rock people (who don’t believe real people use em-dashes) to see AI everywhere and get paid for it?