| > Then they don't in LLMs too LLMs don't get drunk . If a child answers questions from a book of answers then they'll appear to understand the domain insofar as those questions appear. They do not. They will fail to answer questions under, eg., permutations of words (say, a question asks about "norepinephrine" but the book only contains "noradrenaline" etc.). Insofar as a human cannot answer questions under trivial linguistic permutations then they too do not understand the domain. But these are not the kinds of failures experienced with those who have some capacity, eg., for counter-factual reasoning about their environment's physics. In those people it is environmental illusion and cognitive impairment -- not trivial permutations of phrasing which lead to catastrophic loss of apparent understanding. Cognitive impairment = reasoning machine is broken Environmental illusion = data is ambigious and actions cannto resolve it These "failure modes" are expected if you actually have the relevant capacity. |
Well actually they sort of can...
https://www.reddit.com/r/LocalLLaMA/comments/13vv941/tempera...