[autoexec]

> the obvious fact that LLMs are getting better at reasoning than humans

I wanted to say that you were wrong, that LLMs can't reason and so it certainly isn't an obvious truth that they do it better than humans, but when I asked AI if LLMs can reason it told me that they can't which (while still not being reasoned by the LLM) seems to support the spirit of your claim since it gave a correct answer while you (a presumed human that can reason) got it wrong.

[foxglacier]

We might be elevating the importance of reasoning too much because us humans need to use it to solve many difficult problems. But if intuition was stronger, conscious/explicit/logical reasoning might not be needed. Didn't the famous mathemetician Ramanujan say that God gave him his answers in his dreams? That sounds like really powerful intuition like an LLM. Us humans can already solve a lot of incredibly complex problems intuitively, but they're quite domain-specific, like for spatial navigation and social interaction.

[ahazred8ta]

Anthropologist Gregory Bateson predicted we'll know machines are conscious when we ask a question and the computer responds, "That reminds me of a story."

[umanwizard]

How are you defining “reasoning”?

[autoexec]

That seems to be the hangup. I have to use a definition that would put it on equal footing to what we do as humans since that's the comparison being made.

Computers and software can be said to "understand", "think", and "reason" in their own way and informally people have always used those words in that context. Recently, software which has been trained on human-reasoned output is producing text that mimics reasoning well enough that it can be confused for the real thing, but nobody has been able to show that any reasoning (as a human reasons) is what's occurring.

[eru]

Why do you care if the software 'reasons'?

If the output it produces is as useful to me as the output produced by a human with the magical and expensive capability to 'reason', why should I care?

[umanwizard]

You didn't answer my question.

[autoexec]

There are several that would apply. Let's use this one as an example: Reason is the capacity of consciously applying logic by drawing valid conclusions from new or existing information, with the aim of seeking the truth.

[barrkel]

I don't think you need consciousness to reason. I don't see why repeated application of rewrite rules to extrapolate logical conclusions from antecedents shouldn't be considered reasoning. LLMs are perfectly able to match and apply rewrite rules, while using fuzzy concepts rather than being bound to crisp ontologies that make symbolic reasoning impractical to scale up. And for better or worse, LLMs can also apply simplified heuristics and rules of thumb, and end up making the same mistakes that humans make.

[umanwizard]

> consciously

What does this mean?

[autoexec]

If you think "consciously" is a loaded term, wait until you get to "truth"!

Maybe it'd be easier to try another definition:

2 a(1) : the power of comprehending, inferring, or thinking especially in orderly rational ways : intelligence

The same source defined intelligence as:

a(1) : the ability to learn or understand or to deal with new or trying situations : reason also : the skilled use of reason

And here we get the core of the issue. AI doesn't "think". It doesn't comprehend or understand what it does. There is no actual "I" in AI that didn't come from the people whose works were used to train it. At least not yet. I question if LLMs will ever be capable of anything more than producing a convincing affectation of the process used to produce the material it was trained on. I suspect that AGI will have to come from elsewhere. That doesn't mean that what passes for AI these days can't be useful, but I don't think it's capable of reason and as far as I know, nobody has proved otherwise.