[mtlb]

Yeah, the "logical reasoning" in LLMs is mostly a marketing device to get products sold and papers published. One could hope that starting with reasoning instead of trying to get it "emerge" would do a better job. But if we have little idea of how abstract thinking actually works, this is a problem. What do you think about current logic-based AI approaches? Do they try to replicate the best ideas we've got from congnitive sciences, or trying to do their job for them?

[dragonwriter]

> One could hope that starting with reasoning instead of trying to get it "emerge" would do a better job.

We did AI starting with reasoning (directly implementing rules of propositional logic) first, it is called expert systems.

It works very well for some things, but after some efforts to expand it with things like fuzzy logic it became pretty much accepted that we'd reached its limit.

You could hope that it would work better, but...

[nicklecompte]

There's a distinction between "constrain the AI's output with logical rules to make it more reliable" and "build logical reasoning into the AI." The current strategies are trying to do the first task and I bet it'll lead to all sorts of cool technology. I strongly doubt these techniques will extend to the actual logical reasoning. Intuitively, it feels like throwing a bunch of logical rules onto AI is begging the question - I doubt bird/mammal brains actually have these logical rules baked in, I am sure it's far more sophisticated.

A trivial theorem in logic gives an example of what I mean:

If A then B <=> If (not B) then (not A)

This is really not how humans think - I don't believe we have a "contrapositive calculator" in our brain that takes arbitrary situations in and computes a contrapositive. This contrapositive theorem is a fact of the world that humans used logical thinking to understand, and which can be applied to formal logical computations that human brains aren't necessarily good at.

Specifically, I don't think non-human animals have "logical" thinking at all, they have causal thinking, and human logic is a consequence of us having exceptionally good understanding of causality. Logic is itself a special case of causality, formalized in a "generic" fashion by human language and used as a tool to help us think through tricky cases.

The contrapositive theorem takes a bit of thought for me to unwind - "so if B is not true then of course A can't be true" - but the way contrapositives are reflected in the real world takes no thought whatsoever, even if the examples are more algebraically complicated than A->B <=> (~B)->(~A):

- if the door is working and I have a key that can unlock the door, then if I can't unlock the door either I don't have the key or the door is broken. (AvB)->C <=> (~C)->(~A ^ ~B)

- if having gas implies my car can drive, then if my car can't drive I don't have gas - or possibly I was incorrect and my car is broken. (A->B <=> (~B)->(~A)) V (~)(A->B)

These cases are obvious to us because the brain has access to much fancier causal reasoning than what we can currently express in human language. For now, human language is stuck with "If a then not b" stuff. I don't think feeding this limited human language into a computer is going to burst past these limits. We need to figure out how bird/mammal brains actually model things causally.

[mtlb]

> These cases are obvious to us because the brain has access to much fancier causal reasoning than what we can currently express in human language. For now, human language is stuck with "If a then not b" stuff.

I don't follow this. Didn't you just express these cases in human language? I understand that in reality we can "grasp" the meaning of a problem of not being able to open the door without expressing or thinking about it verbally, which would be redundant as there would be a lot to say (the key may be broken, the door may be held by someone on the other side, even if the key works we might be trying to push instead of pull, etc, etc.) and any person who has opened doors with keys would likely understand all of this. The problem is not that those things can't be expressed in human language, but the lack of ability to build good conceptual models of the world that encompasses all such knowledge and allows reasoning on it quickly.

[nicklecompte]

I didn't mean the specific cases, I meant the underlying mechanism that our brain uses to reason about these cases. There is something deeper going on that allows us to build rigorous world models from very thin abstractions, which can be applied to a seemingly arbitrary range of problems. It's this rigorous world model which is absent in AI and not currently explained by cognitive science.

In this example, the overall world model is able to easily accommodate "broken door" "functioning door" "key" etc. and come to a specific conclusion about this problem. The specific conclusion can be easily expressed in human language. The world model itself can't.