[Eisenstein]

> ELIZA could be considered a rudimentary Chinese Room, Markov chains a bit more advanced, but LLMs have proven that given enough resources, LLMs can be surprisingly convincing Chinese rooms.

Eliza is not a Chinese room because we know how it works. The whole point of the Chinese Room is that you don't. It is a thought experiment to say 'since we don't know how this is producing output, we should consider that it is just following rules (unless it is human).

> Food for thought: If I use the weights to blindly calculate the output tokens with pencil and paper, are they thinking, or is it a Chinese Room with a HUGE dictionary?

Well, I never conceded that language models are thinking, all I did was say that the Chinese Room is a lazy way of concluding human exceptionalism.

But, I would have to conclude that if you were able to produce output which was coherent and appropriate, and exhibited all signs of what I understand a thinking system to do, then it is a possibility.