[skywhopper]

So, you agree with the point that they’re making and you’re mad about it? It’s important to state that the models aren’t doing real reasoning because they are being marketed and sold as if they are.

As for your question: ‘So what does "sophisticated simulators of reasoning-like text" even mean here?’

It means CoT interstitial “reasoning” steps produce text that looks like reasoning, but is just a rough approximation, given that the reasoning often doesn’t line up with the conclusion, or the priors, or reality.

[syllogism]

What is "real reasoning"? The mechanism that the models use is well described. They do what they do. What is this article's complaint?

[Yizahi]

For example - at minimum reasoning should match what actually happened. This is not even a complete set of criteria for reasoning, but at least a minimal baseline. Currently LLM programs are generating BS in the "reasoning" part of the output. For example ask the LLM program to "reason" how it produces a sum of two numbers and you will see that it doesn't match at all with what LLM program did in the background. The "reasoning" it outputs is simply an extract of the reasoning which humans did in the LLM dataset. Even Anthropic officially admits this. If you ask a program how to do maintenance on a gearbox and it replies with very well articulated and correct (important!) guide to harvest wheat, then we can't call it reasoning of any kind, despite that wheat farming guide was correct and logical.

[Xss3]

As soon as you introduce multiple constraints on what is and isn't reasoning people get confused and disengage.

I like this approach of setting a minimum constraint. But i feel adding more will just make people ignore the point entirely.

[pxc]

The reality is obvious. The only way not to see it when looking at research like this is to not want to see it. The idea that this critique is somehow more confusing than the use of the word "reasoning" itself is farcical.

LLMs are cool and some of the things they can do now are useful, even surprising. But when it comes to AI, business leaders are talking their books and many people are swept up by that breathless talk and their own misleading intuitions, frequently parroted by the media.

The "but human reasoning is also flawed, so I can't possibly understand what you mean!" objection cannot be sustained in good faith short of delusion.

[intended]

“the mechanism the models us is well described”

Vs

Total AI capex in the past 6 months was greater than US consumer spending

Or

AGI is coming

Or

AI Agents will be able to do most white collar work

——

The paper is addressing parts of the conversation and expectations of AI that are in the HYPE quadrant. There’s money riding on the idea that AI is going to begin to reason reliably. That it will work as a ghost in the machine.

[nerdjon]

This is why research like this is important and needs to keep being published.

What we have seen the last few years is a conscious marketing effort to rebrand everything ML as AI and to use terms like "Reasoning", "Extended Thinking" and others that for many non technical people give the impression that it is doing far more than it is actually doing.

Many of us here can see his research and be like... well yeah we already knew this. But there is a very well funded effort to oversell what these systems can actually do and that is reaching the people that ultimately make the decisions at companies.

So the question is no longer will AI Agents be able to do most white collar work. They can probably fake it well enough to accomplish a few tasks and management will see that. But will the output actually be valuable long term vs short term gains.

[ponow]

I'm happy enough if I'm better off for having used a tool than having not.

[intended]

Most people weren’t happy when the 2008 crash happened, and bank bailouts were needed, and a global recession ensued.

Most people here are going to use a coding agent, be happy about it (like you), and go on their merry way.

Most people here are not making near trillion dollar bets on the world changing power of AI.

EVERYONE here will be affected by those bets. It’s one thing if those bets pay off if future subscription growth matches targets. It’s an entirely different thing if those bets require “reasoning” to pan out.

[throwaway173738]

The scary thing about ML isn’t that it’s poised to eat a lot of lower-reasoning tasks, it’s that we’re going to find ourselves in a landscape of “that’s just what the AI said to do” kind of excuses for all kinds of bad behavior, and we’re completely unwilling to explore what biases are encoded in the models we’re producing. It’s like how Facebook abdicates responsibility for how users feel because it’s just the product of an algorithm. And if I were a betting person I’d bet all this stuff is going to be used for making rental determinations and for deciding who gets exceptions to overdraft fees well before it’s used for anything else. It’s an enabling technology for all kinds of inhumanity.

[sdenton4]

"the reasoning often doesn’t line up with the conclusion, or the priors, or reality."

My dude, have you ever interacted with human reasoning?

[d4rkn0d3z]

Are you sure you are not comparing to human unreason?

Most of what humans think of as reason is actually "will to power". The capability to use our faculties in a way that produces logical conclusions seems like an evolutionary accident, an off-lable use of the brain's machinery for complex social interaction. Most people never learn to catch themselves doing the former when they intended to engage in the latter, some don't know the difference. Fortunately, the latter provides a means of self-correction, the research here hopes to elucidate whether an LLM based reasoning system has the same property.

In other words, given consistent application of reason I would expect a human to eventually draw logically correct conclusions, decline to answer, rephrase the question, etc. But with an LLM, should I expect a non-determisitic infinite walk though plausible nonsense? I expect reaaoning to converge.