[Filligree]

There is no black box 'reasoning' component in humans either.

I will grant you that humans are far more intelligent, and after spending thousands of hours playing with LLMs, it's hard not to see their limitations. At the same time... they're dumb like a very dumb person who has (implausibly) read the Library of Congress, not like a rock or a computer.

I often use Claude to write short stories, largely just for fun. Certainly, its skill at English vastly outmatches its skill at reasoning. It doesn't write well, but it regularly produces turns of phrase that makes me laugh; meanwhile, it needs hand-holding to successfully handle situations with asymmetrical knowledge. It's bad at theory of mind.

But it's just bad, in more or less the same way that a two-year-old is bad at it.

Not the best reasoner in the world. It would be false to claim it's as smart as the typical seven-year-old...

It's almost as wrong to claim that it can't reason at all.

[MrMcCall]

But it's not reasoning, it's just wordplay, just a gargantuanly complex level of auto-generated ELIZA.

[Filligree]

It definitely feels like reasoning. Problems get solved. They may be simple problems, but it's still far beyond what a calculator can do.

Does it really matter if it's "just wordplay"? I'm not convinced humans are any different, beyond the sheer scale. I certainly don't believe we have a 'reasoning module'.

[MrMcCall]

You don't know the history of ELIZA, do you?

That story goes way deeper than some wordplay fooling people. The entire intent was to get people to realize that it was worthless, but, even after people learned that and what it was, they clamoured for more!

"Just imagine how stupid the average person is, and them remember that half of them are even stupider than that!" --George Carlin

And, yes, we humans are very different, but you'll have to traverse my recent comment history to get the extensive explanation. It's worth it, though, I promise, but I doubt you'll like it or agree with it. Good luck!

[genericspammer]

You raise good points, I agree it feels like it is reasoning at times.

Though the brain, with our current understanding of it, is by far more of a black box to us than any LLM.

> I certainly don't believe we have a 'reasoning module'.

Let’s also point out that human brains probably don’t have any vector databases in them either.

It seems to me like our brains must work very differently - just look at how much energy an LLM consumes compared to our brains consuming around 12 Watts.