[Yizahi]

Because it is, for now. For a while at least. You can prove that LLM doesn't understand what it does and it is surprisingly simple. Request it to add two integers and then ask it to explain how it arrived at that result. The answer will be completely unrelated to the actual process LLM used because both results were generated independently and without understanding their meaning and connection.

[lgas]

This is likely true for the majority of humans too.

[Yizahi]

Objectively untrue. Any human who can add two integers or use a knife to cut food or write a word with a pen can afterwards describe what he did at least in some way. Unless he is lying which is a separate topic, we assume an honest attempt. If I wrote a word with a pen via execution motions with my hand, I wouldn't describe it as "I levitated the pen by manipulating gravity with my mind". Or if I added two integers, I wouldn't say that "I created a a lookup table of a many loosely adjacent numbers (different from the numbers in a task) and run statistical analysis on them and did a few more things like that a in a loop". No, I would say that I either calculated sums of decimals, or I I did a school technique with rounding up and then subtracted that adjustment later, or anything which actually happened. If I used a Python sum() in a CLI I would also say that I used exactly that not the other method. LLM can't do it.