[mjburgess]

> Reasoning is a cognitive process

You could have a non-cognitive view of reasoning, or an embodied one. NNs are a-cognitive systems, they do not engage in reasoning of any form.

Reasoning concerns inference across truth-apt propositions (eg., A->B, A thef. B). NNs have no propositions, nor are any parts truth-apt, true or false. NNs are statistical systems which select answers by weights found from optimisation. No process either in optimisation or prediction is an inferential one in the sense of cognition.

I also deny that the "world" as used by frankly just philosophically incompetent ML researchers, whose gross lack of familiarity with basically anything outside pytorch, is even relevant to the sense of "world" that propositions bare a truth relation to.

A world in the relevant sense isn't the state space of the training data -- this is an insane supposition which makes the claim "AI has world models" actually circular. The relevant sense of world is the cause of the training data. If the training data is about an abstract game then you collapse the distinction since the rules are the data.

The famous "NNs learn WMs" paper is just this: choose a system whose data is just a restatement of the system; rather than a measure of a world.

NNs do not form representational models of the cause of their measurement data because all they do is induce (ie., compress by function-fitting) across the measurement space. They model the measurements not their causes. This is only "predictive" of features of the causal origin of measurement data in rigged scenarios, and in general, fails catastrophically to be predictive.

Consider running a NN across photos of the sky: it is impossible for this process to produce newton's law of gravity. The weights are just models of the pixels, and these are not distributed according to this law.

Worse, in general, there is no function from the measurement space to properties of its causal origin -- so it is impossible to build representations by induction. (eg., there is no function Photo->Cat|Dog, the distributions of pixels in photos is ambiguous, and changes over time).

Reasoning, as in cognition, is an (logically) inferential process which considers propositions that bare a truth relation to the world which is the causal origin of concepts which the proposition comprises (created by a biogenerative process). It is the activity of an agent with an interior subjectivity and ecological rationality. Reasoning is done by an agent about something of interest to that agent, with motivation towards a goal the agent has, in the service of the agent's preferences, etc.

If reasoning is an abstract pattern, then rice falling to the ground is likewise "reasoning".

[FrustratedMonky]

This is kind of ignoring other NN's. You're very focused on LLM's as the example.

AlphaGo learned by playing itself. And is able to anticipate multiple moves ahead.

Then, that same 'engine', was able to be applied to Chess, and learned how to beat a master from scratch, by playing itself, in just a few hours.

There was no lookups, or zip'ing of aggregated data.

A lot of what you are postulating as cognition, humans don't do either. Humans didn't figure out gravity from photos of the sky (unless you mean tracing stars and planets, and then yes an NN probably can figure it out). Many humans go through their whole day without analyzing propositions and inferring reality and what to do next. Humans are similarly un-conscious.

A lot of AI news all the time. This link is from today. A little more in the theme of 'world building models', than this current post about LLM's.

https://news.ycombinator.com/item?id=39692387

There was another on also about an NN that could pass some international Geometry competition. It was based on propositions, and reasoning.

[mjburgess]

Yes, it is impossible for anyone to figure out gravity by induction. That's the problem with AI.

The way we build explanations is largely by reasoning-by-analogy. We build physical models with our hands, to resolve ambiguities in our environment, ever more complexly -- and then, eventually, land upon the right analogy that then falls away.

Prior to gravity we had crystal spheres -- reasoning by ananlogy with such things.

Since machines arent in the world, as in my robot example, they can never build explantory conceptualisations of it.

Chess et al. are not worlds in any relevant sense. No one cares, or doubts, that a system with a fully mathematically specific "world" can be "learnt" by a computer.

The full specification of a "world" in mathematical terms doesnt require intelligence. At that point you can use the dumb strategies of alphago.

Intelligence is what you do when you don't know what you're doing. The "World Model"s we're interested in are those that arent already specified to the machine.

All these formal games are just outcomes spaces where every event is known a priori.

As I said, the AI people arent even operating in the category of intelligence. It just the profound lack of background knowledge of any field outside pytorch.lol() that their meglomania plausible

Go look at every paper in AI or ML that purports to build a model of anything: can you find a single one where the outcome space cannot be fully specified either formally (as with chess, etc.) or empirically (as with data samples)?

This has nothing to do with intelligence. We do not either start from the answers, or samples of the answers. We have no answers.

The resolution to this problem requires having a body: you have to move in order to think, in direct causal contact with the world beyond the capacities of clay, to think about it.

[FrustratedMonky]

Guess this is the crux:

"The full specification of a "world" in mathematical terms doesnt require intelligence. At that point you can use the dumb strategies of alphago."

I fall in camp that humans are just glorified amoeba, twitching at stimuli.

Eventually an AI could model 'us' with dumb strategies, because really baked into the human brain/body are just dumb strategies.

[mjburgess]

Dumb is the wrong word, lets say, cheating.

All AI at the moment is just cheating with a fake UI.

We don't learn about the world by first being told what it is like. If you can fully specify an abstract system in mathematics, or use a historical corpus to answer questions --- you're nothing more than a kid cheating.

You seem to think that cynicism requires believing that animals are not, by construction, any different to incredibly absurdly dumb engineered works of our most over hyped morons.

This isnt cynicism, or scepticism, or erudition or sophistication. It's meglomania.

The whole history of evolution has not produced, in us and most animals, the most complex object (, likely,) in the entire universe to do something that alphago is doing. The level of ego here is off the charts.

This view is only a product of a pround ignorance of zoology (and so on) -- and a deep deep anti-intellectualism which says, "reality is easy to know, just build a computer program"

[calf]

To focus on one issue, the neural machine that is chosen by optimization is one that "best" fits the photos of the sky. But those multiple optima do not preclude a neural machine whose parameter values are computationally equivalent to, say, a 3D representation of the sky projected onto a 2D perspective -- a kind of partial world theory or world model, that was picked randomly out of many optima. First, it's not impossible, just highly difficult to find at present technology. Second, the papers describing emergent structures or emergent information inside of actually-existing neural nets point to an empirical possibility that these machines are more than their statistical parts. Both these reasons incline me to stay on the fence on whether neural nets are purely stochastic parrots.

[mjburgess]

> whether neural nets are purely stochastic parrots.

Well we know how they work, it isnt speculative. All gradient-based algs on empirical outcome spaces are just kernel machines (ie., they weight their training data and take averages across it using a similarity metric).

Insofar as the ooutput seems as-if to reason it is because the input was produced by reasoning (of people). If you input text documents which have not been structured by reasoning agents, then the system doesnt work.

As for the idea of AI building generative 3D models and then projecting 2D -- yes, indeed that's how we did it.. but there are very large infinitities of 3D models all producing the same 2D.

This is where the "start from known outcome spaces" strategy of all existing AI fails. You cannot scan an infinity, or even sample meaningfully from it.

In otherwords the AI has to build such "deep models" circumstantially, it has to have a very limited set of them, and these have to 'somehow' be necessarily close to reality.

How do we do this? No mystery, we are in reality and so we in an ecological interplay with our enviorments. THe environment isnt, in cartesian terms, an evil daemon -- it doesnt lie, and doesnt tell the truth. What it does do is act reliably in reaction to us.

Via these means we explain.

[calf]

Or, we don't surely know what deep nets are doing. If I give you an LLM or AlphaGo, you cannot look at it and tell me what it does. It's a bunch of parameters and edge weights. The counterargument is something like, deep nets are overparameterized and the gradient descent process does not reflect the final result. You would think that the large infinities of correct/incorrect 3D models are impossible to choose from, but in practice some have found emergent structural properties - like board positions, formal grammar fragments, etc. - enough to at least suggest that we don't understand how they work, and that it is a conflation/reductive error to call deep nets the same kernel or statistical machines as before.

The above isn't my own argument, as I'm not an expert. But theoreticians have been looking at this, and the ones posing this counterargument come from outside the ML community/Google/OpenAI so you can't attack this argument for being the wild delusions of ML researchers either. The lectures I watched was by an IAS professor in theoretical computer science, not ML people. Another professor's lecture I started watching has a background in signal theory and probability/statistics, if even he says "we don't know what's going on with deep learning", I tend to give that some credence and update my own uncertainty.

Now, I get your argument in that you are repeating everything Chomsky has said regarding explainability, evolution of human cognition and "truth of the world", statistical machines being fed a corpus of human-understandable information be it Internet text or Go game moves. Chomsky's criticism of ML-based "AI" covers all of this and I don't see your argument as introducing anything different from his (feel free to correct me if I've misread your remarks). I myself actually started on his side, now I'm a little on the fence and can see both sides more clearly.

[FrustratedMonky]

sidebar

"If reasoning is an abstract pattern, then rice falling to the ground is likewise "reasoning"."

Technically, I think some philosophers over the ages have made that point, and argued that 'rice falling' is reasoning, and the rice is 'wanting' to be closer to the 'earth' or some such thing. It does sound wacky. Think Leibniz and monads made some argument like that.

And the various takes on 'Will to Power', Schopenhauer argued the will is a blind force. We don't have control of our own thoughts.

I think you are still giving humans too much credit for this aspect of 'cognition'.

"A man can do what he wills, but he cannot will what he wills." Schopenhauer

[mjburgess]

schopenhaur's idealism is indeed very similar, as is all idealism to this computational mysticism.

What none in this tradition considered possible is that the world exists; each reduced it down to a purely formal pattern one way or another.

Thankfully today we treat mental illness, and derealisation and depersonalisation and raised to this status.

I operate in a framework where there's a world and we're in it, and it is one way and not another, and the way it is arises from spatio-temporal properties that arent equivalent because the symbols we use in models of them are isomorphic.

In otherwords, I have passed through my phase of insanity and arrived back into the world where the grass is green because it is green; and the chair heavy, because it is massive,.

[FrustratedMonky]

"Before one studies Zen, mountains are mountains and waters are waters;

After one gains insight through the teachings of a master, mountains are no longer mountains and waters are no longer waters;

After enlightenment, mountains are once again mountains and waters are waters."

------

or another i like

"Before enlightenment; chop wood, carry water. After enlightenment; chop wood, carry water.”

---------

Yes. I do get where you are coming from.

guess after doing a lot of meditating on 'no-self'. I started picking apart my own mind, and realizing it is all just electrical sparks and chemicals.

But then my engineer mind jumped in and said, of course we can model this. So then I get into arguments on the internet about how, of course we can model this.

Lets say we both agree on the real world existing. I think with us living in this real world. We are just arguing over the degree to which we'll be able to model a human. Of course, a model being an approximation. And I lean pretty far in the direction that once we model 'us' sufficiently to be indistinguishable from real 'us', then it will have also generated some subjective experience. (I firmly believed this until just recently after reading 'blindsight', i'm having doubts).

I think the Idealist from back in Schopenhauer day, were doing their best to describe the real world. Sometimes it sounds mystical, but that was before (or during) scientific revolution. The terminology is all different, and they didn't know a lot we take for granted. I wouldn't say they were mystics because they don't have todays knowledge.

The whole 'noumenal' world versus 'Phenomenal' world is valuable 'concept'. Our senses and mind only form an internal 'model' of the real world, it isn't the real world. But we can agree water is wet. But by how much. There are lot of studies about how different people perceive objects as moving faster/slower, etc... based on fear or anxiety. Our inner 'perception' isn't 'accurate', it isn't the actual real world. Just like a computers wouldn't be.