Hacker News new | ask | show | jobs
by itishappy 778 days ago
What happens if it hallucinates the <title>?
3 comments
simonw 778 days ago
You catch it. The hallucinated title will fail to match the retrieved text based on the reference ID.

If it hallucinates an incorrect (but valid) reference ID then hopefully your users can spot that the quoted text has no relevance to their question.

link
mattyyeung 778 days ago
Two possibilities:

(1) if the <title> contents (unique reference string) doesn't match, then it's trivially detected. Typically the query is re-run (non-determinism comes in handy sometimes) or if problems persist we show an error message to the doctor

(2) if a valid <title> is hallucinated, then the wrong quote is indeed displayed on the blue background. It's still a verbatim quote, but it is up to the user to handle this.

In testing when we have maliciously shown the wrong quote, users seem to be easily able to identify. It seems "Irrelevant" is easier than "wrong" to detect.

link
bradfox2 778 days ago
Galactica training paper from FAIR investigated citation hallucination quite thoroughly, if you havent seen it, probably worth a look. Trained in hashes of citations were much more reliable than a natural language representation.
link
resource_waste 778 days ago
Same thing when a human hallucinates.

Except with LLMs, you can run like 10 different models. With a human, you owe $120 and are taking medicine.

link
pton_xd 778 days ago
Except with a human there's a counter-party with assets or insurance who assumes liability for mistakes.

Although presumably if a company is making decisions using an LLM, and the LLM makes a mistake, the company would still be held liable ... probably.

If there's no "damage" from the mistake then it doesn't matter either way.

link
KaiserPro 778 days ago
> With a human, you owe $120 and are taking medicine.

Well there are protocols, procedures and a bunch of checks and balances.

The problem with the LLM is that there isn't any, its you vs one shot retrieval.

link
resource_waste 778 days ago
Step 1: Be born to a physician dad

Step 2: Have your physician dad get you a job at a hospital

Step 3: Have your physician dad's physician friend write a letter of recommendation

Step 4: Get into medical school

Step 5: Have your physician dad reach out to friends at various residencies.

Step 6: Get influenced by big pharma, create addictions, make big money.

link