[shafyy]

> Do you have a theory about when certain LLM capabilities emerge?

We do know how LLMs work, correct? We also know what they are capable of and what not (of course this line is often blurred by hype).

I am not an expert at all on LLMs or neuroscience. But it is apparent that having a discussion with a human vs. with an LLM is a completely different ballpark. I am not saying that we will never have technology that can "understand" and "think" like a human does. I am just saying, this is not it.

Also, just because a lot of progress in LLMs has been made in the past 5 years, that we can just extrapolate the future progress on this. Local maxima and technology limits are a thing.

[uh_uh]

> We do know how LLMs work, correct?

NO! We have working training algorithms. We still don't have a complete understanding of why deep learning works in practice, and especially not why it works at the current level of scale. If you disagree, please cite me the papers because I'd love to read them.

To put in another way: Just because you can breed dogs, it doesn't necessary mean that you have a working theory of genes or even that you know they exist. Which was actually the human condition for most of history.

[shafyy]

We do know in general how LLMs work. Now, it's of course not always possible to say why a specific output is generated given an input, but we do know HOW it does it.

To translate it to your analogy with dogs: We do know how the anatomy of dogs work, but we do not know why they sometimes fetch the stick and sometimes not.