|
|
|
|
|
|
by dagss
25 days ago
|
|
|
Thank you that was clearer. So, I have a PhD in Astrophysics so I am not a total stranger to doing some thinking. And I would say "create (...) answers by relying on the abstract concepts of the objects they are using" is a lofty goal for humans, something to aspire to more than something that typically goes on. We go by habits and intuition and allegories and quite muddy concepts most of the time. Concepts are malleable and evolve in clarity. And in creating new mathematics etc., intuition, inspiration, "flashes of insights" etc after absorbing oneself in the problem has an important role. Are these things we have in our minds, whether concepts or habits or intuitions or flashes of insights, better or worse than whatever patterns could potentially be found in the LLM weights? I struggle to label one of them "understanding" and the other not, at least without involving consciousness somehow. Obviously you can define "understand" as "understand as a human would" but that is circular and uninteresting. We just have to agree to find each others position incredible I am afraid :) |
|
|
Those are already concepts. For LLMs, the word X is just an object linked to the words W, Y, Z, with no meaning to it. Habits, intuition and allegories are using objects to which we are attributing meaning.
Just to clarify, the links that LLMs create are complex, for example depend on all the surrounding other words, but they are still meaningless. If 2 totally different semantic sets of words happen to have exactly the same graphs of links, then you can swap the sets of word together, it does not matter for the LLM. To use a simplified example where you reduce a set to just 2 words, if "garden" and "pea" are linked the same way that "quantum" and "mechanic", then the relationship are the same for the LLM, without the LLM understanding that "garden pea" is a different concept than "quantum mechanic".
That's what I mean by "understanding": humans understand "garden pea" and "quantum mechanic" as concepts (even if they don't know biology or physics enough to even explain how they work), LLMs just use these objects as meaningless entities. All there is is a graph of links used to generate a sentence, but without knowing what the sentence means. A bit like if someone was giving you all the words of a language you don't know and the exact rules of how to build an answer given an input, but that you don't know what each word means.
Of course, the relationship learnt by the LLMs are very complex, allowing big changes based on the surrounding other 100'000 words. But learning this relationship is still more straightforward than to leap into a conceptual world model (especially because there is nothing to guide the LLM. If "garden pea" and "quantum mechanic" have the same graph geometry, then the world model where "garden pea" is an abstract field of study and "quantum mechanic" is a material object is as probable than the opposite).