[autoexec]

That seems to be the hangup. I have to use a definition that would put it on equal footing to what we do as humans since that's the comparison being made.

Computers and software can be said to "understand", "think", and "reason" in their own way and informally people have always used those words in that context. Recently, software which has been trained on human-reasoned output is producing text that mimics reasoning well enough that it can be confused for the real thing, but nobody has been able to show that any reasoning (as a human reasons) is what's occurring.

[eru]

Why do you care if the software 'reasons'?

If the output it produces is as useful to me as the output produced by a human with the magical and expensive capability to 'reason', why should I care?

[umanwizard]

You didn't answer my question.

[autoexec]

There are several that would apply. Let's use this one as an example: Reason is the capacity of consciously applying logic by drawing valid conclusions from new or existing information, with the aim of seeking the truth.

[barrkel]

I don't think you need consciousness to reason. I don't see why repeated application of rewrite rules to extrapolate logical conclusions from antecedents shouldn't be considered reasoning. LLMs are perfectly able to match and apply rewrite rules, while using fuzzy concepts rather than being bound to crisp ontologies that make symbolic reasoning impractical to scale up. And for better or worse, LLMs can also apply simplified heuristics and rules of thumb, and end up making the same mistakes that humans make.

[umanwizard]

> consciously

What does this mean?

[autoexec]

If you think "consciously" is a loaded term, wait until you get to "truth"!

Maybe it'd be easier to try another definition:

2 a(1) : the power of comprehending, inferring, or thinking especially in orderly rational ways : intelligence

The same source defined intelligence as:

a(1) : the ability to learn or understand or to deal with new or trying situations : reason also : the skilled use of reason

And here we get the core of the issue. AI doesn't "think". It doesn't comprehend or understand what it does. There is no actual "I" in AI that didn't come from the people whose works were used to train it. At least not yet. I question if LLMs will ever be capable of anything more than producing a convincing affectation of the process used to produce the material it was trained on. I suspect that AGI will have to come from elsewhere. That doesn't mean that what passes for AI these days can't be useful, but I don't think it's capable of reason and as far as I know, nobody has proved otherwise.

[barrkel]

Comprehend, from com- ("together" or "with") and prehendere ("to seize" or "grasp"). To take a hold of.

Can a calculator comprehend arithmetic? Can it take a hold of a number (in a register, for example), and a second number, and add them together to get a hold of the result?

What is computation, really? When we design machines to do arithmetic, do the machines actually do arithmetic, or do they just coincidentally come up with states that we humans can interpret as a correspondence with arithmetic?

More importantly, would a rose by any other name smell as sweet?

If you put a problem into text, and give it to an LLM, and an LLM applied a series of higher order pattern matching to it to produce more text, and you read the resulting text and interpret it as reasoning about the solution to a problem, has the LLM reasoned? Does the calculator calculate? Or does it really matter?

[int_19h]

Have you ever watched an LLM with CoT solve a logical puzzle?