[dzikibaz]

How are "LLM hallucinations" different from a low-quality training dataset or randomly picked tokens due to overly random sampling settings?

[citrin_ru]

What I see even in good models is that when you ask something hard or impossible (but looking routine) instead of replying “I cannot” they hallucinate. A better dataset would help only to solve problems which can be solved (based on this dataset).