[garyclarke27]

Zuck did this on purpose, humiliating LeCun so he would leave. Despite LeCun being proved wrong on LLMs capabilities such as reasoning, he remained extremely negative, not exactly inspiring leadership to the Meta Ai team, he had to go.

[aiven]

But LLMs still can't reason... in a reasonable sense. No matter how you look at it, it is still a statistical model that guesses next word, it doesn't think/reason per se.

[astrange]

It does not guess the next word, the sampler chooses subword tokens. Your explanation can't even explain why it generates coherent words.

[levocardia]

It is insane to think this in 2025 unless you define "reasoning" as "the thing I can do that LLMs cannot"

[ActorNightly]

Reasoning is the act of figuring out how to solve a problem for which you have no previous training set. If an AI can reason, and you give it a standard task of "write me a python file that does x and y", it should be able to complete that task without ever being trained on python code. Or english in general.

The way it would solve that problem would look more like some combination of Hebbian Learning and Mu Zero, where it starts to explore the space around it interms of interactions, information gathering, information parsing, forming associations, to where it eventually understands that your task involves the action of writing bytes to a file in a certain structure that when executed produces certain output, and the rules around the structure that make it give that output.

And it will be able to do this through running as a model on your computer, or a robot that can type on a keyboard, all from the same code.

LLMs appear to "reason" because most people don't actually reason - a lot of people even in technical fields operate on a principle of information lookup. I.e they look at the things that they have been taught to do, figure out which problem fits closest, and repeat steps with a few modifications a long the way. LLMs pretty much do the same thing. If you operate like this, then sure LLMs, "reason". But there is a reason why LLMs are barely useful in actual technical work - under the hood, to make them do things autonomously, you basically have to specify wrapper code/prompts that take often as long to write and finetune as actual code itself.

[aiven]

It is insane to think this in 2025 unless you define "reasoning" as some mechanical information lookup. This thinking (ironically) degrades the meaning of reasoning and of intelligence in general.