[adamc]

Quoting from the blog post:

"The inability of standard neural network architectures to reliably extrapolate — and reason formally — has been the central theme of my own work back to 1998 and 2001, and has been a theme in all of my challenges to deep learning, going back to 2012, and LLMs in 2019."

I think he makes a pretty lucid point that people have been questioning this for a long time, and definitely longer than 3 years. If you think there is some particular feature of LLMs that makes this a temporary hurdle, maybe you should make that point.

[ajross]

I think I did make the point, but I'll do it again. Teaching a LLM to reason is 100% isomorphic to teaching a child to reason. All the logic being deployed here by the luddite set[1] could be deployed to explain why your grade schooler will never reason correctly. And it's wrong there, and there's no reason to expect that it's wrong here.

Very broadly: you learn to reason by learning to write and run "code" in your head. Can an LLM write and run code? Yes, it can. Do they use it currently to "reason" well? No, because no one has made that work yet. Does that constitute an argument that they CANNOT? Clearly not.

[1] And I'm no LLM booster! See the point about the pendulum upthread.