[roxolotl]

This is an interesting point but while I agree with the article, don’t think LLMs are more than sophisticated autocomplete, and believe there’s way more to human intelligence than matrix multiplication humans also cannot explain in many cases why they did what they did.

Of course the most famous and clear example are the split brain experiments which show post hoc rationalization[0].

And then there’s the Libet experiments[1] showing that your conscious experience is only realized after the triggering brain activity. While it’s not showing you cannot explain why it does seem to indicate your explanation is post hoc.

0: https://www.neuroscienceof.com/human-nature-blog/decision-ma...

1: https://www.informationphilosopher.com/freedom/libet_experim...

[Yizahi]

I agree, but here we are veering into more complex decision making. I was talking about much simpler cases, like for example going through a handful of simple steeps for simple task. For example addition, ask a person to sum two numbers and then ask to explain what he just did step by step and a person would be able to do it. Person may even make a mistake in process but the general algorithm will be matching what actually happened. Query LLM for the same, and while LLM answer will be correct for a human, it won't match what LLM actually did to calculate. This is what outs LLM "thinking" for me, they just generate a very plausible intermediate steps too.