Hacker News new | ask | show | jobs
by ubj 1290 days ago
To an extent yes, but this can quickly become overwhelming.

For example, editors and reviewers for academic journals / conferences will likely see a deluge of AI-generated "scientific" papers. Their time is limited, and odds are that more papers with incorrect information will slip through the peer review process.

To be clear, peer review today certainly isn't perfect at catching bad papers. But AI generation has the potential to exaggerate the problem.

2 comments

That's already been a problem for some years already:

https://dailysceptic.org/2022/06/08/fake-science-the-threat-...

The sad thing is it doesn't take a ChatGPT level intelligence to beat scientific peer review. Journals routinely publish papers that are completely auto-generated gibberish. A simple generative grammar or template splicer is apparently enough. These are articles that are immediately visible as the work of a program at first glance, they wouldn't make it past even the most amateur blog or student newspapers, yet they surface in their thousands in journals that are supposed to be the epitome of accurate knowledge!

Worse, the journal publishers are doing nothing about it. Their current approach to trying to fix the problem is to try and use the work of random CS academics to make "spam filters" for paper submissions. The more obvious solution of having editors and reviewers who actually read scientific papers before they are published appears to be rejected out of hand.

For inspiration, here is how the NYTimes deals with anonymous sources:

What we consider before using anonymous sources: How do they know the information?

What’s their motivation for telling us?

Have they proved reliable in the past?

Can we corroborate the information they provide?

Because using anonymous sources puts great strain on our most valuable asset: our readers’ trust, the reporter and at least one editor is required to know the identity of the source. A senior newsroom editor must also approve the use of the information the source provides.