[FrustratedMonky]

"The pattern isn't the important bit."

I was taking pattern as the electrical impulses in the brain. Which I would say is everything when it comes to thought and thinking.

Human Neurons are just Calcium Voltage potentials. Just like weights in a NN.

Yes. The human brain neurons are far more complicated than that. There is a lot of chemical soup of hormones and modulators that factor into the voltage potentials. What you eat can impact gut, that impacts how many neurotransmitters are produced, that impact when a neuron fires, that is experienced as a mood, etc.... SO yes, the human brain cannot be separated from the body.

There is some entity, brain+body. And it so happens that it is made from Carbon . And we call things based on Carbon to be 'organic' and it can re-produce so we call it 'alive'. These are just definitions, that we have assigned to things.

Just like a CPU can't be removed from it's power supply and keep working. There is a 'system' and it has 'parts'.

That doesn't mean, we can't model a brain to a degree close enough that you couldn't tell an AI and Human apart.

I'm just saying, that when we do reach that point, then we'll be forced to realize the AI also has an internal subject experience, and is 'conscious', or we'll have to acknowledge that humans do not.

It will be either/or.

Humans and AI are both conscious, have internal subjective experience. or Neither do. And if neither do, then humans really are just hallucinating their inner experience, but not in control. Humans are also deterministic. Nothing special.

I see from your other posts, that you seem to be putting a lot of faith in Chaos Theory or Quantum Mechanics as some underlying explanation. To give humans the special sauce.

I'd just say, introducing randomness, does not make a system un-determined. That there is a lot of randomness in our 'chaotic' body, does not imbue it with agency. Randomness != Agency.

Edit: I like the clay example. I'd say that is like the NN that goes through training, then released, but the model is static. Doesn't get updated. Eventually these AI's will learn as they go and be continually updating.

[mjburgess]

Talking to people is a proxy measure of their subjective experience, it isn't a direct measure. Since you can talk to a tape recorder and, so long as it plays back responses, you and the tape recorder likewise are engaging as-if conversing.

Since computer science isnt a science, but a form of (applied, discrete) mathematics it encourages people to think in terms of functions with purely mathematical semantics, as if 2 + 2 = 4 were the same thing whether it modelled the divison of cells or the printing of a book.

What causes people to speak is our intentions, desires, theory of mind, representational ability, imagination... etc. We speak because we are in a shared world of social intentions, and we desire to communicate something about ourselves or this world to others.. we translate, clumsily, these features of our experience into text tokens; and hope that the agency we are speaking to can recover our mind from those tokens.

Over a million years we have specialised a culture of communication to enable this illusion to take place: the illusion that meaning is in the patterns of the symbols we use.

You can, of course, build a system to perfectly immitate these patterns; just as a video game, if you hold the viewer fixed, my appear to contain a world with a table and a glass. But if you reach for that glass of water, it isnt there: it's an illusion.

This is all statistical AI is: a trick. It's a replaying back of our own conversations to each other, as if it was a real conversation with us.

We can determine, as certain as you like, that there are no goals, intensions, desires, imagination, counterfactual reasoning -- no body, no observation. The machine is not in the world with us, and it not responsive to the world -- the machine generates text, it does not speak.

You inclination to analogise the machine to a person is just on the grounds that you are strapped into you chair, and observing the video game, believe you can grab the glass of water inside.

I am not strapped into a chair, nor do you have to be. You can do science: you can build real experimental explanations of how we form representations, intentions, goals, desires etc. And it is trivial to explain AI, there is no mystery to "compress reddit and query over its space of text tokens". There is only the illusion that the user is subject to -- the belief that the agency lies in this querying process, and not in the redditors who had cause to speak to each other about their experiences of hte world .

[FrustratedMonky]

Everyone is captivated by the current hot thing, LLM/GPT's.

But LLM's are not the whole of AI research.

A lot of your arguments are based on 'embodied' reasoning. Humans live in the world, they need to eat and survive. LLM's just compress what humans generated in the world. Correct, current LLM's are mostly regurgitating, but they don't "speak because we are in a shared world of social intentions".

I'd say game worlds are the frontier, because they are able to simulate a lower resolution world for current AI's to learn in. And in that world, they do embody it and have purpose (rewards/goals), they need to survive.

DeepMind's AlphaGo was when I switched. https://www.wired.com/2016/03/two-moves-alphago-lee-sedol-re... Move 37, it was called alien, creative, inhuman intelligence.

Now, scale that up to our world, with admittedly, thousands/millions of more variables. Put it in a robot body, with vision (the latest studies show AI vision building a world model context). Add some goals. Bam, some dangerous stuff, AI embodied in the world, with a goal to survive.

It might be far away, but where we are now was supposed to take another hundred years. So who knows.

The military is already running world simulations where the AI's goal function lead it to kill the soldier operating the AI in order to bypass him. It 'learned' to bypass the operator by killing them to achieve its goal.

Yes, Hyperbole. But really, not by much.

But back to our discussion. At that point, does the robot have an internal subjective perspective? Did AlphaGO when it was reasoning about its small low variable world?

[mjburgess]

It never has an "internal subjective experience" -- AlphaGO wasnt reasoning.

Reasoning is a cognitive process in which propositions, which model the world, are considered in turn and subject to ecological rationality (concerns of utilty, effort, interest, preference, etc.).

At no point in the flight of an aeroplane does it ever lay eggs.

You are using smoke to establish fire, these ways of measuring internal mental states of animals only work on animals.

If you can produce a robot with no prior conceptual scheme of, say, a novel apartment it is thrown into; a robot which can then determine what is in that apartment, how roughly it works (eg., light switch -> lights turn on), of an account of that apartment; explain why it has explored it; show that its behaviour is moderated and caused by these stated goals; ask it for opinions about the apartment etc. -- then we are actually playing the intelligence game, at least. Rather than stupid magic laterns.

Now, does this robot have a subjective experience?

Well I think we need to keep going with our tests: does it have an aversion to toxic stimulous? Is this aversion moderating its goals and behaviour? Are its memories contextualised by these aversions (eg., does its process of remembering display a variety when remembering negative vs. positive experiences)? And so on.

If I can ask, "Did you find my apartment fun?" and it can answer because it did, or did not -- then we're very close.

That is if we can show the reason it says, "yes" or "no" had to do with a history of taste, judgement, preference, curiosity, etc. all built up by itself -- not under "supervision with the right answers" but with no problem-specific answers ever given... and so on.

Questions of these kind arent even revenant to anything in AI. Any sincere AI engineer will say that they have nothing to do with the goals of the system theyre building. All AI that we can actually access, 100% has no interest, methods or ambitions to deliver any of the above.

AI isnt even in the category of intelligence; it isnt even trying to produce it.

[calf]

So I'm trying to understand your argument here, but why isn't "Reasoning is a cognitive process" circular logic?

AlphaGo wasn't reasoning, how? I think reasonably AlphaGo has modeled a world, and it is by design subject to the bounded rationality of a game-theoretic optimization problem. So two of your criteria are satisfied.

So - I'm just reading your definition here - AlphaGo wasn't reasoning because it is not a cognitive process. Is that your specific argument? And if so, then what is a cognitive process?

I'm just focusing on this part here and I don't see the argument clearly:

> It never has an "internal subjective experience" -- AlphaGO wasnt reasoning. > Reasoning is a cognitive process in which propositions, which model the world, are considered in turn and subject to ecological rationality (concerns of utilty, effort, interest, preference, etc.).

[mjburgess]

> Reasoning is a cognitive process

You could have a non-cognitive view of reasoning, or an embodied one. NNs are a-cognitive systems, they do not engage in reasoning of any form.

Reasoning concerns inference across truth-apt propositions (eg., A->B, A thef. B). NNs have no propositions, nor are any parts truth-apt, true or false. NNs are statistical systems which select answers by weights found from optimisation. No process either in optimisation or prediction is an inferential one in the sense of cognition.

I also deny that the "world" as used by frankly just philosophically incompetent ML researchers, whose gross lack of familiarity with basically anything outside pytorch, is even relevant to the sense of "world" that propositions bare a truth relation to.

A world in the relevant sense isn't the state space of the training data -- this is an insane supposition which makes the claim "AI has world models" actually circular. The relevant sense of world is the cause of the training data. If the training data is about an abstract game then you collapse the distinction since the rules are the data.

The famous "NNs learn WMs" paper is just this: choose a system whose data is just a restatement of the system; rather than a measure of a world.

NNs do not form representational models of the cause of their measurement data because all they do is induce (ie., compress by function-fitting) across the measurement space. They model the measurements not their causes. This is only "predictive" of features of the causal origin of measurement data in rigged scenarios, and in general, fails catastrophically to be predictive.

Consider running a NN across photos of the sky: it is impossible for this process to produce newton's law of gravity. The weights are just models of the pixels, and these are not distributed according to this law.

Worse, in general, there is no function from the measurement space to properties of its causal origin -- so it is impossible to build representations by induction. (eg., there is no function Photo->Cat|Dog, the distributions of pixels in photos is ambiguous, and changes over time).

Reasoning, as in cognition, is an (logically) inferential process which considers propositions that bare a truth relation to the world which is the causal origin of concepts which the proposition comprises (created by a biogenerative process). It is the activity of an agent with an interior subjectivity and ecological rationality. Reasoning is done by an agent about something of interest to that agent, with motivation towards a goal the agent has, in the service of the agent's preferences, etc.

If reasoning is an abstract pattern, then rice falling to the ground is likewise "reasoning".

[FrustratedMonky]

This is kind of ignoring other NN's. You're very focused on LLM's as the example.

AlphaGo learned by playing itself. And is able to anticipate multiple moves ahead.

Then, that same 'engine', was able to be applied to Chess, and learned how to beat a master from scratch, by playing itself, in just a few hours.

There was no lookups, or zip'ing of aggregated data.

A lot of what you are postulating as cognition, humans don't do either. Humans didn't figure out gravity from photos of the sky (unless you mean tracing stars and planets, and then yes an NN probably can figure it out). Many humans go through their whole day without analyzing propositions and inferring reality and what to do next. Humans are similarly un-conscious.

A lot of AI news all the time. This link is from today. A little more in the theme of 'world building models', than this current post about LLM's.

https://news.ycombinator.com/item?id=39692387

There was another on also about an NN that could pass some international Geometry competition. It was based on propositions, and reasoning.

[calf]

To focus on one issue, the neural machine that is chosen by optimization is one that "best" fits the photos of the sky. But those multiple optima do not preclude a neural machine whose parameter values are computationally equivalent to, say, a 3D representation of the sky projected onto a 2D perspective -- a kind of partial world theory or world model, that was picked randomly out of many optima. First, it's not impossible, just highly difficult to find at present technology. Second, the papers describing emergent structures or emergent information inside of actually-existing neural nets point to an empirical possibility that these machines are more than their statistical parts. Both these reasons incline me to stay on the fence on whether neural nets are purely stochastic parrots.

[FrustratedMonky]

sidebar

"If reasoning is an abstract pattern, then rice falling to the ground is likewise "reasoning"."

Technically, I think some philosophers over the ages have made that point, and argued that 'rice falling' is reasoning, and the rice is 'wanting' to be closer to the 'earth' or some such thing. It does sound wacky. Think Leibniz and monads made some argument like that.

And the various takes on 'Will to Power', Schopenhauer argued the will is a blind force. We don't have control of our own thoughts.

I think you are still giving humans too much credit for this aspect of 'cognition'.

"A man can do what he wills, but he cannot will what he wills." Schopenhauer