[PJHkorea]

This is my personal opinion LLM is essentially a probability model that selects the word (token) with the highest probability of coming next based on context. However, if you start selecting word combinations (branches) with low winning probabilities or unverified outcomes for the 'exploration' of MCTS, it may look plausible the first one or two times, but over time, it generates nonsense (hallucinations) that are completely out of context. In coding terms, this is a phenomenon where you arbitrarily imagine and write a library that does not exist.

[frwrfwrfeefwf]

hallucinations are often good ideas for things that should exist, missing functions or libraries

[PJHkorea]

That's right! Extreme hallucination in a well-aligned state is extremely helpful.