[thu2111]

Even GPT-3 doesn't encode common sense, which is why it can't do a lot of basic physical reasoning. It's "just" word prediction, albeit very impressive word prediction.

If you look at what GPT produces closely, a lot of it is simply bullshit. It sounds plausible but is wrong. That's exactly the wrong type of AI to detect plausible-but-wrong-bullshit papers, which are the most common type.

[pas]

Right, I worded that a bit lazily. There's no confidence score output from GPT-3, but if there were and if the user would select to only get high confidence results then it would shut up quickly. And that's what I meant by common sense. Of course it depends on the corpus. It's really-really just text, as you said. (It's possible that it can somehow eventually encode high level things like arithmetic, but so far it seems, even if it does have that model somewhere embedded, it doesn't know how/when to use it.)

The language model (GPT-3) doesn't have to understand physics, it just have to help extract out some semantics of the paper.

There needs to be a classifier on top trained with a labeled set of good and bad papers.

[thu2111]

I think there is a confidence score actually! Most blogs about it don't show them but this one went into it:

https://arr.am/2020/07/25/gpt-3-uncertainty-prompts/

It's really cool how the uncertainty prompts alter the confidence associated with the next words.

I guess I'm not disagreeing with you in the abstract that a theoretically strong enough AI could identify bad papers, especially if it had some help for 'real' arithmetic. It at least could flag the most basic issues like plagiarism, cited documents that don't contain the cited fact, etc. Detecting claims that are themselves implausible seems like the hardest task possible, however. That's very close to general AI.

[pas]

> Detecting claims that are themselves implausible seems like the hardest task possible, however. That's very close to general AI.

Yes, of course. I was simply trying to say that an AI can be quite successful in detecting the usual no-nos, eg. multiple comparisons without correcting for it, p-hacking, or ... who knows what "feature" the classifier would find. Maybe there's simply none, so it'll be really up to subject matter experts to review them. (But it's unlikely, because there are quite successful blogs devoted to simply picking apart shoddy papers simply based on looking at the controls, and other parts of experiment design and the methods sections, and of course the aforementioned stats.)