[sagebird]

should have submitted it to 5 independent fact checkers - would have deflated nonsense before it began by showing that you are going to see trivial and non trivial shifts among them.

mostly true and misleading counting as separate buckets while also being somewhat orthogonal conceptually is also stupid.

a better output format might be true|false|unknowable, confidence where confidence is 0..1

at least then you can compare agreement among models as a distance measurement and not a moronic bucket agreement

the conclusion is actually obvious: llms are good enough for most of this work and it is definitely cheaper, so you should use llms for fact checking at least as a first pass