| The right word is "confabulation". Which is when we fill in missing information but may not be aware that we are doing it. We all confabulate to some degree, as any neural system must, since no training data is stored perfectly. Human "hallucinations" in contrast, are a particular kind of breakdown in our sensory feedback loops. Which is not a process LLMs even have. Hallucinations occur when our internal sensory feedback loops overpower actual sensory input, resulting in a stream of false sensory experience/signals being generated and processed. The false running experience might still incorporate some actual sensory information or not. When we dream, we are hallucinating - our sensory experience loop running free of our actual senses - to a productive purpose. The reason our senses have feedback is so that we can use our interpretation of sensory input as cues to make interpreting the next moments input easier. But its important that our running interpretation can reset when new input significantly diverges from our expectations so it can quickly reorient. (Not only is it important to revert to a raw input interpretation to ensure our running interpretation keeps up the actual context changes and corrects misinterpretations, but such resets signal that something novel or unexpected has happened, so likely trigger learning.) So "hallucinations" was an unfortunate and misleading choice of terminology. |