Hacker News new | ask | show | jobs
by SilasX 557 days ago
I've found it very helpful to make the following distinction:

Spec: Do X in situation Y.

Correctness bug: It doesn't do X in situation Y.

Fitness-for-purpose (FFP) bug: It does X in situation Y, but, knowing this, you decide you don't actually want it to do X in situation Y.

Hallucination is an FFP bug.

1 comments

Sorry, but it's a correctness bug most of the time[], as the correct information is known or known to not exist.

If ask a math question and you get a random incorrect equation, it's not unfit for purpose, just incorrect.

FFP would be returning misinformation from the model, which is not a hallucination per se. Or the model misunderstanding the question and returning a correct answer to a related question.

[] Except for art generators.

"Correct" here doesn't mean "correct" information -- I made sure to clarify what it means with an example.