[js8]

The article says:

"Reasoning means being able to put those concepts together to solve problems."

There's more to reasoning than just following rules of logic ("putting concepts together"). It is also detection where the concepts cause contradictions and do not fit, and the whole mysterious magic of how to modify the concepts to make them fit.

In the first meaning of "reasoning", AI (and computers) have been able to reason for a long time. It's the second meaning that evades us.

I said before that in the 90s, cutting edge AIs were based on various theories of how to do reasoning under uncertainty (fuzzy logic, bayesian networks, etc.). Then deep NNs blew these systems out of the water in practice, but at the expense of us not understanding how they reason with uncertainty, and if there is any consistency to it. So we progressed, but didn't reconcile this problem, what is the right way to reason with uncertainty, and it might just be very very hard. (That's why I am interested in P vs NP, as I believe there is an answer there.)

[FrustratedMonky]

>"at the expense of us actually not understanding how they reason with uncertainty"

That is the crux. It is doing something to anticipate words beyond the next token. It has to, to construct these long coherent documents. Just like humans do.

Just because we don't understand it doesn't mean it isn't reasoning. Isn't 'thinking ahead'.

Just like I can say we don't understand the brain, thus humans aren't actually reasoning.

There are a lot of brain studies that look into the pre-cursor changes in the brain pre-ceding conscious thought.

We can't say NN aren't doing something similar.

[js8]

> We can't say NN aren't doing something similar.

I agree, but my point was different. We can't say what it does theoretically, therefore we don't know how reliable it is (we don't understand the tradeoffs and failure modes). At least most humans have a way to assess their own reliability, and can see where their reasoning (or of their fellow humans) is inconsistent.

[FrustratedMonky]

There was good thread on this subject yesterday, more about LLM and Language.

It comes down to the physical world.

Humans can " assess their own reliability" when they can all point at something in the real world, and come to some agreement on what they are all seeing, what to call it, etc..

When humans get off base, if it is tangible, like an apple, they can all point at the apple, and bring themselves back into alignment, that is an apple.

But, for abstract concepts in philosophy, or morals, etc.. Something that is not tangible. Humans can 'drift' just as much as AI.

Humans can get into echo chambers -> and 'go nutz', absorbing others misinformation.

LLMs Learning from other LLSm' -> the 'models drift' over time.

https://news.ycombinator.com/item?id=37811610

[kypro]

I'm not sure I'm following. Uncertainty is really just another way of saying probability. You handle uncertainty with probabilistic approaches which neural networks do.

Or put another way a square explicitly might be four straight lines connected at right angles, but in reality such a perfect shape is never going to exist. What's important is that the system understands that shape which has roughly straight lines and roughly connected and right angles is "squarish" enough to be a labeled a square, and the less "squarish" the shape becomes the less certain the system becomes that square is the correct label. Neural networks certainly achieve this.

We might not always understand what the parameters of a neutral networks are encoding, but that's a limitation of our brains, not of neural networks.

[js8]

Probability is common, but not the only way to model uncertainty. There are also different logics, Dempster-Shaefer theory and so on. That aside...

Neural networks can be modeled with probability, but that doesn't mean they actually compute in that way. Just like with humans - we can see brain often follows things like Bayes rule, but it doesn't compute PDFs. Doing full probability reasoning would be too expensive for NNs to do, so they cut corners somewhere, and we don't really understand where, it might be very inconsistent. It often works but also often fails.