[tovej]

Because the humans are reasoning and the LLMs aren't? I have yet to use an LLM for a complex problem and not have it hallucinate.

I expect a reasonable counterargument here would be 'but the LLMs have chain of thought now, and that's reasoning". I disagree, but I think that's a reasonable point of view. I can concede that point because it does not materially change the value of the output. Even if it does use chain of thought, an LLM gives you extremely trite solutions based on probable text, it still has no context in which to reason, it's "reasoning" in platos cave using the shapes of real world objects, filtered through a lossy language model.

LLMs are great for one thing: brainstorming, and brainstorming is only useful if you have no idea what to do in the first place. Once you know _anything_ substantial about the subject matter an LLM loses its value to you as a conversation partner.

[cl3misch]

Are you not reasoning on lossy abstractions?

I'm still not on board with the (seemingly prevalent) notion that LLM's can't reason. What's reasoning, anyway? I'm not actively advocating for any side, but the arguments against reasoning always felt very tautological to me.

[tovej]

The burden of proof is on the argument that they _are_ reasoning, and I have seen very little evidence that they do.

It's also immediately clear to me when I look at the architecture of transformers that reasoning is not in the cards. I could be convinced otherwise if, again, someone showed me an indication of reasoning behavior. Since there is no such evidence and the systems theory approach tells me it does not reasonably reason, I have a pretty darn good reason not to believe it's reasoning.

[cl3misch]

> It's also immediately clear to me when I look at the architecture of transformers that reasoning is not in the cards.

I'm not saying that's incorrect, but thb that's exactly the tautology I was talking about!

[tovej]

That's not a tautology. That's the summary of the argument itself. If you want to know more, then a good reason why it can't be reasoning is that there is no evaluation of the truth value of any statement at any point, only the likelihood of the statement being found in the training set. This evaluation has no relationship with truth.

If no statement is ever evaluated, it's not logical reasoning, because logical reasoning requires the evaluation of truth values of statements.

[cl3misch]

> there is no evaluation of the truth value of any statement at any point

You could argue that the attention part of the network is some form of truth validation of the next predicted token? But indeed, the current chat interfaces don't change their previous text output retroactively.

Still, I am not really convinced. If we assume a human can reason, what does "evaluation of the truth value" mean? Thinking about something? This is still performed with our implicit mental model of the world, coming from shadows on a cave's wall, right?

[FranzFerdiNaN]

People here are hallucinating too. So many people making obviously wrong claims with full confidence, which you only notice when it’s about something you know a lot about yourself.

[timmy-turner]

They do but we have for instance education to reduce their hallucinations in narrow fields of expertise. And a system of guardrails to only let educated people work in those fields to avoid harm.

[gonzobonzo]

The above person was comparing it to friends and random online comments, though. I wouldn't be surprised if AI is far more reliable than those.

[macrolime]

This is like saying:

Gogole is great for one thing: brainstorming, and brainstorming is only useful if you have no idea what to do in the first place. Once you know _anything_ substantial about the subject matter Google loses its value to you.