| > Firstly, how do you know that the optimal way to highly compress complex information is to understand it? What is your non-performance baseline for "Understanding"? We don't have such a measure for humans. Understanding is the behavioral ability demonstrated by learning to model something complex well. Beyond mappings, associations, interpolations. Models clearly do. Mix up the most unlikely combination of non-trivial subjects, and they response sensibly. Those are not averaged, interpolated by any order, or even combinatorially interactions. There is a reason those kinds of encodings, mappings, associations, interpolations, statistics / stochastics, all failed miserably for decades. Still fail. It took topological transforms, reminiscent of how we compute (dendrite-soma-axon, tensor-sum-nonlinear), and then they lept several orders of magnitude ahead of any alternative. The problem with models composed of relationships of lower order than the phenomena they are trying to model, is they require combinatorially more parameters to model anything complex. For simple problems, poor models fail gracefully. For complex problems, poor models just fail. |
How do you even know it is the case?
How do you know the output is not the result of combinatorial interactions?
How do you even know that the "sensible" response on unlikely combination is not the result of a simple recipe that "make the response sounds sensible"? Either you, yourself, have some expertise on the subject, and therefore the combination does probably exist in the AI training data, or you don't and you have no idea if the response is sensible or is the usual smooth talk that everyone could come up spending 2 or 3 hours googling on the subject and crafting something sensible.
Worse, you are saying that the model "understand", which means that it discovers the underlying mechanism that drive the output. This "understanding" is a set of equation that link different concept, that explain how one concept affects another concept. So, it is "combinatorial interaction". Not a simple linear one, but guess what, LLM are designed to introduce non-linearity.
Even when AI are able to find new solution of math problem, the result is, like when done by humans, by using existing basic tools to build more complicated ones.
> It took topological transforms, reminiscent of how we compute (dendrite-soma-axon, tensor-sum-nonlinear), and then they lept several orders of magnitude ahead of any alternative.
And yet, the LLM elements that are "similar" or "analogue" to how the human brain works are very small. The human brain has thoughts "flowing", while LLM can only work "by step". The human brain is able to learn on a very reduced dataset, while LLM need more data that a human will ever be able to analyse, even less store. The human brain has "memory" and "context" intrinsically intertwined with how it works, while you can decouple these from the LLM. ...
Finally, here is a good contradiction of having you in one side saying that AI is mimicking the human brain and it is why it works well and on the other hand saying that AI will find the lowest minimum and that this minimum is "understanding how the phenomenom works" rather than "repeating by hearth what it was told during training".
As a human, when you mentally compute 6 times 7, what do you do? Do you do: "6 follows 5, which follows 4, which follows 3, ... and 7 follows 6, which follows 5, ... so we have (1 + 1 + 1 + 1 + 1 + 1) times (1 + 1 + 1 + 1 + 1 + 1 + 1), which is 1 + 1 + 1 + 1 + ..."? I guess you probably don't, you just remember the most helpful element you remember by heart. For example, you remember by hearth that 6x7 is 42. Or you remember that 3x7 is 21, and therefore 6x7 is the double, 42. Or you remember that 7x7 is 49, and therefore 6x7 is 42. Or even have a "feeling" from a mixture of all these (6x7 is somewhere around 40 because 5x7 feels like being around 30 and 7x7 feels like being around 50, and if I think of number in the 40 that "feels" like they are from the 7-multiple-table, I remember 42).
Same thing when a human does 324x42: the majority of humans will decompose it in "simpler" multiplication that they remember by hearth and, and only then, they will combine them. It is a good example of how the brain optimise: by balancing the trade-off of "using memory" and "using understanding": basic operations use memory, but of course it is inefficient to use memory for all numbers, in which case it will use a combination of both.
The way human do basic math operation is not purely by "understanding" arithmetic, it is by relying on what they remember from their training. At the same time, humans know how arithmetic works, and they will use it when relevant. Yet, the human brains prefer to rely on some "learnt by hearth" elements. This is in contradiction with your assertion that optimisation will always lead to "understanding" and that human brains is optimizing the same way AIs do.
This is only one example with numbers, but of course it works with plenty of other things. This is also exactly why humans get "the wrong idea" on plenty of phenomenon, that are then described as "counter-intuitive".
The reason "by hearth" is part of a good strategy rather than "purely understanding" is because there is a trade-off between "memory" and "compute", in both the human brain and AI: it is easier (and therefore a stronger attractor during the optimisation of the process of "getting the correct answer") to do the faster operation "retrieve from memory" than to do the slower operation "retrieve the theory from memory, compute the first step, store it in the short term memory, compute the second step, store it in the short term memory, compute the final answer by adding the first step answer and the second step answer".