[atoav]

Asking ChatGPT for advice is like asking a genius trickster for advice. There is a chance what they say is 100% true, but the chance that the trickster confidently tells you something entirely made up is definitely there.

And that is the main issue. I can trust some friend or co-worker to tell me when they don't know, because they like me and they have no incentive to tell me bullshit — yeah they might look better if they did, but if there is any doubt on my side it just makes them look bad and that this happens sooner or later is nearly certain.

ChatGPT has no concept of this, so it will happily just give you something plausible but wrong.

[latexr]

However—some defenders will say—your friend or co-worker may think they know the answer but be wrong and still give you the wrong information confidently.

To which I’d respond that it’s important to not ignore the continuity of life. The person giving you the information may themselves learn they were wrong and let you know later and unprompted. Or you may learn the facts and tell them, thus correcting it for everyone else they share with later. In addition, you’ll have a mental note of the friends and coworkers most suited to ask about particular subjects, maximising your changes of getting a right answer.

[lowercased]

The unprompted part, that's the important bit. If you find out you were wrong in with previous recommendation or advice or just basic info, let me know. Email, text, whatever. I don't think there's enough AI agent ecosystem to support that yet.

[appplication]

It seems like the product for this could be a separate model that evaluates statements for truthiness (possibly some other NLP model that correlates to a known source, as well as checking that it doesn’t contradict another known source). This maybe doesn’t have to be an LLM (maybe indeed is a completely independent architecture). LLM output could be run through this model as a final filter, with strictness perhaps defined by user parameters or use case.

It certainly isn’t an easy problem, but it does feel as though it’s structurally solvable. Defining trusted sources vs standard training data seems to be a first step, as well as identifying a means of segmenting domains under which those sources would carry authority.

[lowercased]

[deleted]