[jbritton]

Some have realized that LLMs don’t really reason and that LLMs may not be as amazing as the claims. However, I think LLMs plus agents, plus advances that are likely to come may very well prove to be more valuable than anyone can foresee. It’s very difficult to predict the future profitability of tech.

[fl7305]

> LLMs don’t really reason

Do you have a test for this?

Or is it based on the presumption that reasoning skills cannot evolve, it can only be the result of "intelligent design"?

[jbritton]

I have been spent many hours with them on coding tasks. As things currently stand once context or complexity reaches a certain point they become completely incapable of solving problems and that point occurs on very simple things. They appear completely brain dead at times although they are magnificent liars at making you think they understand the problem. Although, I recently got chatGPT 5 to solve a problem in a couple hours that Claude Sonnet 4 was simply never going to solve. So they are improving. I don’t know the limits. I’m more hopeful that a feedback loop with specialized agents will take things much further. I’m extremely skeptical that getting bigger context windows and larger models is going to get us reasoning. The skepticism comes from observations. Clearly no one knows how thinking actually works. I don’t know how to address the evolve part. LLMs don’t directly mutate and have selective pressure like living organisms. Maybe a simulation could be made to do that.

[jbritton]

Gotham Chess has done chat bot chess championships. The chat bots make a few good moves, and then begin making illegal moves, randomly removing or adding pieces, and completing ignoring obvious threats and attacks. It is so obvious that the pattern matching is not resulting in reasoning. Another example is Towers of Hanoi. An LLM can write code to solve it, because that’s an easy pattern match. But it can’t write out the steps beyond a 3 disk puzzle. It has no understanding of the recursive nature of the problem.

[fl7305]

>> the presumption that reasoning skills cannot evolve

> I don’t know how to address the evolve part. LLMs don’t directly mutate and have selective pressure like living organisms.

Sorry, that was poorly worded. I meant "can reasoning skills not be evolved through the neural net training phase?"

Sure, once you deploy an LLM, it does not evolve any more.

But let's say you have a person Tom with 5-minute short term memory loss, meaning he can't ever remember more than 5 minutes back. His reasoning skills are completely static, just based on his previous education before the memory loss accident, and the last 5 minutes.

Is "5-minute Tom" incapable of reasoning because he can't learn new things?

> They appear completely brain dead at times

Yes, definitely. But they also manage to produce what looks like actual reasoning in other cases. Meaning, "reasoning, not pattern matching".

So if a thing can reason at some times and in some cases, but not in other, what do we call that?

An LLM is a lot like a regular CPU. A CPU basically just operates step by step, where it takes inputs, a state memory, and stored read-only data, puts those into combinatorical logic to calculate new outputs and updates to the state memory.

An LLM does the same thing. It runs step by step, takes the user input+its own previous output tokens and stored readonly data, puts those into a huge neural network to perform processing to generate the next output token.

The "state memory" in an LLM (=the context window) is a lot more limited than a CPU RAM+disk, but it's still a state memory.

So I don't have a problem imagining that an LLM can perform some level of reasoning. Limited and flawed for sure, but still a different creature than "pattern matching".

[whattheheckheck]

Can you post your experiments please?

[jbritton]

I don’t have a log of work that I can post.

[handoflixue]

Would you agree that many humans, especially younger ones, are also incapable of reasoning?

[addaon]

I think it’s completely uncontroversial that younger humans are incapable of reasoning, no? The only area for discussion is at which age (if any) this changes for an individual.

[hattmall]

I would say that's a pretty controversial take unless you strictly mean babies. At the point a kid can walk they are certainly capable of some level of reasoning because just walking successfully prior to it becoming muscle memory requires a fair amount of reasoning and children very rapidly figure out new things both spatial and logical.

[addaon]

So if you believe babies do not reason, and walking humans do, then younger humans up to some age do not reason, yes?