[feoren]

The understanding is inside of the system, in LLMs and in the Chinese Room. I agree with Daniel Dennett that it's preposterous to say that Chinese is not understood in any meaningful sense in the Chinese Room scenario -- it's just that the understanding has been hidden away in the background of the scenario.

Language is tremendously complicated. "Time flies like an arrow, but fruit flies like a banana." "Hard hats must be worn on site; dogs must be carried on escalators", etc. Predicting the next token requires understanding, full stop.

> if the rules are followed, no understanding is neccessary.

The rules are the understanding.

(Note that understanding != consciousness)

[overgard]

Autocomplete can predict the next token without understanding. It's a matter of probability. LLM's predict the next token with much more accuracy because they have much more data on token spatial relationships. To me that's not "understanding", that's "impressive statistical pattern discovery". The difference between a machine and a human though is we can infer that pattern discovery from a small data set, whereas LLM's can only do that because they have the entire internet's worth of data and then some. Of course you're going to make pretty good predictions of text if you have all the knowledge humanity has ever created. I'd be more impressed if you could make a small language model that could reason from first principles to become a large language model.