If you've never had an LLM (all models) suddenly start spouting nonsense in a completely different language...you haven't been using LLMs that much. They will go absolutely insane some % of the time.
They can “go insane” but it seems often to be infra related as opposed to anything one would consider hallucination. Smaller models will often get stuck repeating a word or phrase forever but that’s a bit different and nobody would call it hallucination.
One annoying one is we have an LLM-as-a-judge that is supposed to quote parts of a transcript to justify its reasoning, and sometimes it’ll get stuck on something short like “No.” and then just endlessly repeat it: “4. No. 5. No. […] 728. No. […] 1435. No. …”
They can “go insane” but it seems often to be infra related as opposed to anything one would consider hallucination. Smaller models will often get stuck repeating a word or phrase forever but that’s a bit different and nobody would call it hallucination.