[simonmesmith]

So, some of the findings in relation to the psychology of AI fact-checking they found may be valid and interesting (e.g. how it increases confidence, even in false information when incorrect). But their methodology to me seems quite flawed:

1. They use “ChatGPT” and never mention if it’s 3.5 or 4 (unless I missed it—don’t think so) so I suspect 3.5.

2. They ask ChatGPT to evaluate the truthfulness of new headlines outside its training data cutoff, don’t show it the source of the headline (e.g. a reputable news outlet or not), and don’t allow it to to use an internet search to determine if the news is corroborated.

So their claim that it’s “ineffective” seems like a pretty big overreach. At best you might say “AI fact-checking by relatively weak models not given all available information and asked to evaluate headlines outside their training data cutoff” are ineffective. Duh!

Clearly, if you were to design a good AI fact-checker, you wouldn’t design it like this. You would give it all available information, allow it to corroborate with a news search, and use a stronger model or a model fine-tuned for the task.