[thfuran]

I'm not sure that limit is meaningful. Suppose that there's a system with the approximate reasoning ability of a person, but it doesn't forget things and it has studied every textbook ever written. Would you say that it's only as intelligent as a person?

[dougmwne]

Yes, I am trying to separate out complexity of reasoning from knowledge base and speed. A language model is like a person who has access to Google and several hours or days to research a response. Even a model with poor reasoning, ability can create answers faster than the most intelligent human on the planet, it’s just the nature of computational speed. And a model can be trained on the sum total of every book ever written, and every word ever published to the Internet but that doesn’t make it by itself.

One very key difference I see is that language models can’t create something absolutely novel. They would not have been able to invent calculus if it wasn’t in the training set, while it was possible for a few very smart humans to do such a thing.

[famouswaffles]

Language models can generate novel functioning protein sequences that didn't exist before nevermind in the dataset.

https://www.nature.com/articles/s41587-022-01618-2