|
> ... is a lofty goal for humans, ... We go by habits and intuition and allegories and quite muddy concepts most of the time Those are already concepts. For LLMs, the word X is just an object linked to the words W, Y, Z, with no meaning to it. Habits, intuition and allegories are using objects to which we are attributing meaning. Just to clarify, the links that LLMs create are complex, for example depend on all the surrounding other words, but they are still meaningless. If 2 totally different semantic sets of words happen to have exactly the same graphs of links, then you can swap the sets of word together, it does not matter for the LLM. To use a simplified example where you reduce a set to just 2 words, if "garden" and "pea" are linked the same way that "quantum" and "mechanic", then the relationship are the same for the LLM, without the LLM understanding that "garden pea" is a different concept than "quantum mechanic". That's what I mean by "understanding": humans understand "garden pea" and "quantum mechanic" as concepts (even if they don't know biology or physics enough to even explain how they work), LLMs just use these objects as meaningless entities. All there is is a graph of links used to generate a sentence, but without knowing what the sentence means. A bit like if someone was giving you all the words of a language you don't know and the exact rules of how to build an answer given an input, but that you don't know what each word means. Of course, the relationship learnt by the LLMs are very complex, allowing big changes based on the surrounding other 100'000 words. But learning this relationship is still more straightforward than to leap into a conceptual world model (especially because there is nothing to guide the LLM. If "garden pea" and "quantum mechanic" have the same graph geometry, then the world model where "garden pea" is an abstract field of study and "quantum mechanic" is a material object is as probable than the opposite). |