[simonw]

Something I've found surprisingly effective is telling ChatGPT to "use credible sources" - you can then watch its thinking trace and see it do things like ruling out random blogs, considering media publications with a good reputation for fact checking, and double-checking information that seems unlikely.

[vunderba]

For any LLM with web tool usage I’ll often suggest that it pull relevant papers from arXiv, with an expectation that it cite sources. I find that this helps improve accuracy and lets me go deeper into the material if I want to.

[westoque]

I don't think random blogs should be ruled out. Just because a blog is random doesn't mean its untrue.

[simonw]

The reputation of the writer matters a lot. Recent models seem to be pretty good at evaluating author credibility as well.

What's dangerous is treating information from an anonymous author as factual. You gotta check that in other ways.

[bulbar]

It should not be seen a credible source though.

It can be, but that's hard to correctly judge on at scale.

[skorniienko]

Love it, I'll run tests for "credible sources" and will compare the results. Could be a first feature-request implemented!

[skorniienko]

Added this as issue so I don't forget https://github.com/SerhiiKorniienko/bullshit-detector/issues...