[Apocryphon]

> The article can tell "Cort was informed about the new techniques from a family member back from Jamaica". How do you verify that? The only way is to redo the all work from scratch, refound all the historical references and cross-validate each of them. It's a huge work.

Since this is about checking textual references, as opposed to laboratory work, would it be possible for an LLM to do that? Seems like the hardest thing would be for the A.I. to log into the requisite gateways for the databases hosting the papers.

[cauch]

I doubt it is so naively simple as "querying a database". For example, before the publication of this article, the majority of the "databases" were simply saying "the inventor is Cort", which is one example of thing that the A.I. will get trivially "wrong" when reviewing the paper.

And if it is true, then this A.I. review method can also be applied to mathematics, theoretical physics, computer science, ...

And even if it is doable, you will still have things that will pass the review process when ideally it should not. It is just an hard limitation, same as the one in "justice" (impossible to not sometimes judge a guilty person "innocent" or an innocent person "guilty", anyone who thinks otherwise just don't understand how complicated it is)

[nvm0n2]

It probably can be done, but does the academic establishment really want word to get out that their great research is being auto-reviewed because the "peers" were so bad at their job that an AI can do better? They've spent decades pushing the idea that peer review is a gold standard on the world, it would be quite a climbdown to admit ChatGPT can do it better.