[ajross]

It seems like the needle is now swinging too far back, pointing to "LLMs will NEVER work". And I don't think that's very grounded either.

All these criticisms are valid for human beings too. That kind of question trickery trips up school kids all the time. It's hard to use our brains to reason. It takes practice, and the respresentation of the "reasoning" always ends up being alien to our actual cognitive experience. We literally have invented whole paradigms of how to write this stuff down such that it can be communicated to our peers.

So yeah, LLMs aren't ever going to be "better" at humans at reasoning, necessarily, simply because we both suck at it. But they'll improve, likely via a bunch of analogs to human education. "Here's how to teach a LLM about writing a formal proof" just hasn't been figured out yet.

[lukev]

I don't even think that's what this is doing, though. As technologists, the point isn't to be bullish or bearish on LLMs... we should be focused on empirically understanding what they can do, and why, so that we can design systems to leverage them most effectively and work around their areas of weakness.

This article is important for that because it helps articulate the limit of what (current) LLMs can do. Even if you're an AI maximalist, it's essential to understand the current areas of weakness to design better models or build systems that compensate.

[adamc]

I struggle to see the use of this comment. Many human beings have jobs where they reason about problems far more complex than this every day. Sure, not every human is great at this. But the interest in using LLMs as agents does kind of require that they can routinely get this right -- the author of the blog post mentions this explicitly.

[ajross]

> Many human beings have jobs where they reason about problems far more complex than this every day.

And they hold degrees from decades of education that taught them how to do that. Kids, even smart ones, can't do this reliably. I have two.

I'm just saying that 3 years into the AI Revolution is a bit premature to demand that they "routinely get this right" when you yourself took probably 20 years to get to that point.

To be blunter: this discourse has a very I Am Very Smart vibe to it, which seems pretty amazingly ironic.

[adamc]

Quoting from the blog post:

"The inability of standard neural network architectures to reliably extrapolate — and reason formally — has been the central theme of my own work back to 1998 and 2001, and has been a theme in all of my challenges to deep learning, going back to 2012, and LLMs in 2019."

I think he makes a pretty lucid point that people have been questioning this for a long time, and definitely longer than 3 years. If you think there is some particular feature of LLMs that makes this a temporary hurdle, maybe you should make that point.

[ajross]

I think I did make the point, but I'll do it again. Teaching a LLM to reason is 100% isomorphic to teaching a child to reason. All the logic being deployed here by the luddite set[1] could be deployed to explain why your grade schooler will never reason correctly. And it's wrong there, and there's no reason to expect that it's wrong here.

Very broadly: you learn to reason by learning to write and run "code" in your head. Can an LLM write and run code? Yes, it can. Do they use it currently to "reason" well? No, because no one has made that work yet. Does that constitute an argument that they CANNOT? Clearly not.

[1] And I'm no LLM booster! See the point about the pendulum upthread.