[falcor84]

I'm not following, what's "by definition" here? You can verify the integrity of an AI report in the same way you would do with any other report someone prepared for you - when you encounter something that feels wrong, check the referenced source yourself.

[sgt101]

I think it's normal to invest authority in a report that someone else has prepared - people take what is written on trust because the person who prepared it is accountable for any errors... not just now, but forever.

LLM's are not accountable for anything.

[falcor84]

That's a lofty ideal, but the typical research report is flawed in multiple ways, and even in academia "Most Published Research Findings Are False" [0], and very few people's careers have in any way suffered, as there's very little de-facto accountability for this trust.

The best case scenario as I see it, is that we become more critical of research in general, and start training AI to help us identify these issues. And then we could perhaps utilize GAN to improve report generation.

[0] https://en.wikipedia.org/wiki/Replication_crisis

[falcor84]

P.S. There's a paper from today demonstrating the effectiveness of AI identifying errors in human research: https://news.ycombinator.com/item?id=43295692