[sigmar]

>So what context would cause me to seriously consider the possibility that engineers had created a computer program that is conscious and an intentional user of language? Let me outline one potential sequence of steps. The first requirement is that the computer program has a body (either physical or virtual) and sense organs; there are many reasons for this, but for the purposes of this discussion the most relevant one is the fact that without a body, a computer program could have no desires or emotions, and I believe desires and emotions are necessary for consciousness. Then I’d want to see an embodied agent that could navigate its environment in order to survive as well as, say, a lizard can (and as a point of comparison, certain iguanas can live for decades in the wild). Next I would want to see an embodied agent with the same capacity to deal with novel situations as a mouse. After that I’d want to see agents whose social dynamics are as complex as those of wolves, and then agents with the tool-making abilities of chimpanzees. At that point I would want to see people successfully teaching such embodied agents how to communicate their desires, perhaps by using a button board or some other nonlinguistic modality, the way that people have taught chimpanzees and domesticated dogs.

I agree with some parts of this piece, but paragraphs like this one above seem pretty uninspired and simplistic. It's entirely plausible that a conscious mind would not be evolutionarily incentivized to be able to do those things. ie just because animals on earth needed to develop specific talents doesn't mean that other conscious entities need to. Why would a computer program need to hunt for food like a mouse would? Making tools like chimp? these seem like nonsensical metrics.

[sdenton4]

Yeah, it's kind of mind boggling that Ted Chiang (of all people!) can't imagine intelligence without a body. and the whole thing just begs a lot of questions.

Is a car a body? Does an AI situated in a car therefore get to have desires and emotions? Is a taupe box with a webcam attached a body? (For that matter: Is a quadropelegic body a body? Do quadropelegics have desires and emotions? Obviously, yes and yes.) Why is a body necessary for the formation of desires and emotions? Why are desires and emotions necessary features for consciousness?

Or here's one: If I'm not experiencing any particular emotion in a given moment, am I concious?

I tend to think that emotions, at least, are mainly hormonal global triggers: they're more about physiology than actual consciousness. The whole thing, as a result, sounds like an effort to privilege biological intelligence, rather than a real foray into the issues.

[pasquinelli]

> Yeah, it's kind of mind boggling that Ted Chiang (of all people!) can't imagine intelligence without a body.

the subject is consciousness, not intelligence.

[gabrieledarrigo]

> Yeah, it's kind of mind boggling that Ted Chiang (of all people!) can't imagine intelligence without a body. and the whole thing just begs a lot of questions.

How can you have a subjective experience without a body?

That's the point, in my opinion: your physical/chemical state (body) in a given moment is then translated into the higher abstraction of the emotion. An emotion that *you* feel, because you are self aware of what's happening.

How can you be self aware without feeling? And how do you feel, without a body?

[grumbel]

> How can you have a subjective experience without a body?

Easy, you don't have subjective experiences because you have body in the first place. You have them because some signals come in from your nerves, which your brain turns into a world model. You are effectively a "brain in a vat", the vat just happens to be placed on top of your body.

An AI system constructs the world model a little different, by all the text that gets feed into it, but that doesn't mean that there is anything fundamentally different in the world model it builds. Consciousness operates on world model, not on the world or even the body itself.

The AI's world model might be missing some information, because they weren't described in enough detail in text, but that shouldn't matter for consciousness. A blind or deaf person isn't less conscious than one that can see or hear just because some information is missing from their world model.

[felipeerias]

Your brain is part of your body, that's the point. There isn't a "you" separate from your actual, physical existence. Mind-body dualism is not real.

[grumbel]

The "you" isn't your physical body, it's the pattern recognition in your brain that that classifies some parts of its sensory input as "you" (see Mirror Hand illusion). That it happens to be running on meat instead of silicon is an implementation detail and not important.

If we want to know if AI is conscious or not, we have to ask if the AI can recognize itself in the input it gets.

Some aspects like limited content length and lack of ability for the model weights to update will certainly limit what the AI can do. But that's ultimately a matter of degree, not kind, when it comes to consciousness.

[Jensson]

> The "you" isn't your physical body, it's the pattern recognition in your brain that that classifies some parts of its sensory input as "you" (see Mirror Hand illusion). That it happens to be running on meat instead of silicon is an implementation detail and not important.

It is important, if I destroy your brain and grow a new one and start running the same program on the new brain your consciousness is still dead and its a new person living.

Computer AI models don't run on a single machine, they run in a distributed manner using different machines at different times. When you ask a follow up question thats not sent to the same machine, its another machine answering. So the consciousness one of those computing machines experience would be extremely fragmented and not at all conscious about any discussion with you, since it only saw a few fragments of it.

And no consciousness doesn't expand to cover larger distributed computations, otherwise social media would have developed a consciousness by now but it hasn't. Groups of humans don't start to share consciousness, it doesn't happen, so you can assume groups of distributed computers wont as well.

[solomonb]

Its amazing that after years of advocating for a materialist view of the mind, the tech bros are flipping to mind-body dualism now that they need to believe a concious mind can exist no body at all.

[Planktonne]

It's worse than that; it's mind-body dualism when arguing that an AI can be conscious, but still materialist when arguing that humans are simply more complex neural nets. It's not a coherent viewpoint.

[Gooblebrai]

> You have them because some signals come in from your nerves, which your brain turns into a world model. You are effectively a "brain in a vat", the vat just happens to be placed on top of your body

I think we give too much credit to the brain. The gut has almost as many neurons as a dog's brain and the heart has neurons too. "You" are more likely the whole ecosystem, not just your brain. There are even some hypothesis of disorders like depression being more influenced by the gut than the brain.

[sfn42]

Its a computer program. It is literally just a lot of zeroes and ones, sitting there doing nothing.

Then a request comes in, and the system does a bunch of calculations using those bits, and spits out a result. The bits are unchanged.

When your brain receives input, it is changed. It is constantly active. If it ever stops being active it's dead.

So, what exactly is the claim? Are the bits constantly conscious? Do they snap into consciousness when the computer does math with them? Or is it maybe the computer that's conscious while it's processing these bits? How about when it stops doing that and goes back to doing other stuff? Why are these particular bits special? Was the computer always conscious?

I feel like the only way anyone could believe LLMs are conscious is if they don't understand how computers work. Of course it isn't conscious, how could it possibly be conscious? Its literally just bits. It's like saying the text in a book is conscious.

[orangecat]

So, what exactly is the claim? Are the bits constantly conscious? Do they snap into consciousness when the computer does math with them? Or is it maybe the computer that's conscious while it's processing these bits? How about when it stops doing that and goes back to doing other stuff? Why are these particular bits special? Was the computer always conscious?

These are all fine questions, and they don't become any easier to answer if you replace "computers" with "brains" and "bits" with "neurons".

Of course it isn't conscious, how could it possibly be conscious? Its literally just bits.

"That's ridiculous. How can meat make a machine? You're asking me to believe in sentient meat."

[el_jay]

> These are all fine questions, and they don't become any easier to answer if you replace "computers" with "brains" and "bits" with "neurons".

What is even being argued here - neuroscience is hard, so programming your PC thus makes it conscious?

> "That's ridiculous. How can meat make a machine? You're asking me to believe in sentient meat."

… what?

[nprateem]

> if they don't understand how computers work

Or if they're retards. The fact this still comes up is weird. A printing press isn't conscious, so why would an LLM be.

Don't forget, some of the bros are overly excitable. Like that twat who reckoned a Google model 5 years ago was conscious.

[hn_acc1]

If you feed the same prompt to the same AI with the same random seed, you'll get the same answer every time.

If a hundred people see the same event, will they all respond the same?

[Jtarii]

If I ask my friend what his favourite song is 100 times I will get the same answer back 100 times. Is my friend therefore not conscious?

[Jensson]

> If I ask my friend what his favourite song is 100 times I will get the same answer back 100 times.

Pretty sure you wont, humans vary the exact wording. They will say the same song but they wont answer the exact same way every time. Even if they say the same words two times they wont use the same tone and body language, as they don't just communicate via words and that nonverbal language is a part of what we say.

[jcattle]

If you ask him 100 times in a row I would bet that by the second, maybe third, time he will not answer with his song but with: what the fuck is wrong with you?

Nicely circling back to LLMs not being able to learn and form memories.

[grumbel]

> If a hundred people see the same event, will they all respond the same?

If you have hundred different people, they will of course do something different. Just like hundred different AI model will do something different. The question you have to ask is if the same person under the same circumstances would do the same.

Luckily, we have an answer to that: They would. Transient Global Amnesia is a condition where people temporarily lose the ability to form memories and in turn they keep repeating the same conversation again and again[1]. Their brain keeps asking the same question again and again, as it doesn't remember the answers it already got.

[1] https://www.youtube.com/watch?v=N3fA5uzWDU8

[keeda]

Have they had the exact same experiences and influences all the way to that point in time?

[gabrieledarrigo]

> Easy, you don't have subjective experiences because you have body in the first place. You have them because some signals come in from your nerves, which your brain turns into a world model.

Bro, I think we discarded this idea from Platone and Cartesio a while ago...

Your brain is your body.

Your mind is not detached from it, and you can't feel anything, and so have a subjective experience, without it. Neither, your mind, or "soul" could survive to the physical death of your body.

So...I mean...

[grumbel]

The point is that the brain operates on information, not things. The thing you think is the external world, that's just electricity flowing through your neurons. When you see a dog, that isn't a thing in the world, but a signal in your brain. In principle we can take a knife and cut that connection at any point and replace the real signal with an electronic box that gives the same signal.

You don't need a body, you need electrical signals your brain interprets as body. And in principle you don't even need a brain, you could replace that with some matrix multiplication or transistors that do the same stuff.

The important part of consciousness is being able to figure out what of the sensory input is correlated to your own action and which was caused by the rest of the world.

[gabrieledarrigo]

> You don't need a body, you need electrical signals your brain interprets as body.

As I was saying, your brain is literally part of your body. It's a living organ of your body that receives those inputs, as you say, from the rest of your body.

> And in principle you don't even need a brain, you could replace that with some matrix multiplication or transistors that do the same stuff.

I completely disagree here, and that's why guessing that a bunch of matrix multiplications are coscient and have self awareness of their own experience while running in a CPU seems quite hilarious to me.

But hey, that's just my opinion.

[Tadpole9181]

You need to draw that thinking out to it's natural conclusion, though. If I cut out your brain and stopped you from hearing or seeing or feeling - you would still be a conscious human being capable of thinking and awareness.

If I hooked up electrodes to the hearing centers of your brain and force fed you dialog you perceive as speech (but is really a great deceiver), then responded in what you thought was speech (but are really just probes I use to convert your thoughts to text), that wouldn't suddenly be less real to you. It wouldn't devalue your sapience.

[goatlover]

How do you know the brain separated from the rest of the nervous system and body would still be sapient, capable of thinking and awareness? There's an assumption you're making that the brain is all that's needed, but the nervous system extends throughout the body. One can argue sensory organ are part of the nervous system.

Embodied cognition rejects this assumption. We didn't evolve as brains that were then put in bodies, we evolved as bodies with nervous systems.

[gpderetta]

While dreaming, the brain synthetizes sensory experience while cutting down (or greatly suppressing) most of external stimuli. Yet it is still conscious.

[felipeerias]

Mind-body dualism is not real. Even in your example, you would be building upon a minimal part of a person's body.

[margalabargala]

Your point generalizes to, your emotional state is a reflection of the state of your physical medium.

Why can't that physical medium be GPUs and RAM? And temperature sensors and cameras? What's special about our meat that it's our "body" in the way a computer is not the body of an AI?

I don't think the point being argued can be true without some incredibly contrived, human centric definitions of "body".

[felipeerias]

What are you, ultimately, if not your body?

[margalabargala]

I don't know. I also don't see how that changes the question. What is an AI, ultimately, without its computer hardware?

[felipeerias]

Hardware is fungible. Each LLM response in a conversation could be served from a different machine.

Would you be "you" in a different body?

[JoshTriplett]

> How can you have a subjective experience without a body?

If every neuron of your brain were simulated precisely on a sufficiently powerful computer, that simulation would have subjective experiences, without having a body.

[sfn42]

Citation needed.

Also that's impossible. It is impossible to simulate reality exactly using digital computers. The best we can do is approximate. Doesn't matter how powerful it gets, it'll always just be an approximation.

[acyou]

What does "simulate exactly" mean? To me, exact simulation is not so much an impossibility as a nonsensical concept. What subset of reality are we simulating, and to what degree of precision, and with what certainty? An "exact" precision as it relates to real world objects is not a well understood or defined concept. For integers, I can say there is exactly one Earth orbiting exactly one Sun, but I think that statement is riddled with assumption, inaccuracy and imprecision. For example, it is assumed that I am referring to the present Earth, but is the statement of when the statement is made or when it is heard? Even the word "is" is inexact.

[sfn42]

I agree, exact simulation is nonsensical which is my whole point. Did you read the comment I replied to?

It is nonsensical to claim that anything other than my brain could produce the same consciousness that my brain is producing. It's obviously far beyond anything any conceivable digital computer could ever reasonably simulate, and even if you did create a "good" simulation it obviously wouldn't have the same properties that my brain does because it's an entirely different thing than my brain is.

[JoshTriplett]

Assuming you don't believe humans have any metaphysical component, then the only remaining question is whether there's some essential component to being human that depends on impossible-to-precisely-simulate portions of reality. Nothing we currently know of biology suggests that that would be true, as much as it continues being pursued by people who need there to be something mysterious about consciousness or brains.

In any case, a closely-but-not-perfectly-accurate simulation of a real human brain is still going to be human, unless you believe that someone becomes less human when they're experiencing some kind of cognitive decline, or a stroke, or other biological malfunctions. The point is, there is nothing essential to the having of a physical brain that creates the concept of consciousness or sense-of-self.

[sfn42]

How can a simulation be human when it isn't human? It's a simulation. A human is a human, a simulation is not. Anything that is not a human, is not a human and can never be a human.

And there absolutely is something essential to the concept of being human that we are entirely incapable of replicating artificially. In fact as far as I know we are incapable of synthesizing any kind of life whatsoever. We can't even create the simplest type of living cell imaginable.

So to claim that we could just create consciousness, a fundamental property of this "life" thing that we don't properly understand, within a piece of rock is beyond naive. We don't even know what it is or how it is.

[lupire]

Every computer/program I have ever used has a body and sensors (afferent)and actuators (efferent).

So I have no idea what distinction Ted Chiang is trying to draw

[martin1975]

> How can you have a subjective experience without a body?

Great q. Deepening it further-how can you have a subjective experience without consciousness, which isn't necessarily tied to physicality. Taking it one step further-can you have consciousness without a mind? Who's the first mind, the first cause of it all, that begot both the material and immaterial world?

Fun stuff eh?

[thinkling]

A lot depends on where you draw the boundary for consciousness. Michael Pollan (in his new book, _A World Appears_) distinguishes simpler sentience from more advanced consciousness, and the requirements for sentience (be aware of sense data, have preferences, be able to respond to senses appropriately) are met by plants and single celled life (e.g. moving up a nutrient gradient). Recent findings in plant science are particularly mind blowing. Some are in Pollan's book, more are in _The Light Eaters_.

[pizzly]

How can you have a subjective experience without a body?

Some anecdotal data.

Many dreams I have are just of the computer screen of some coding problem. I think the problem could be x, so I try x. But I don't type the keyboard or anything, the code just magically appears as soon as I think of the solution. Then run the code (but no clicking) and it works or not. I feel in the dream success feeling or failure feeling but there is no body at all.

Also I have other dreams where there is no body that I am aware of but not going there in public.

There is no body sensation in these dreams. But dreaming is very much being consciousness as well as feeling emotions. So answering your question its possible to have a subjective experience without a body but whether you needed the body to learn to have that sensation without a body in the first place is unanswered.

I suspect sensory inputs are more important than a body. If that is the case then eyes can be replaced with cameras, ears with microphones etc. Text input is just another sensory input.

[confidantlake]

What was the thing that was dreaming? The brain. Your brain is part of your body.

[pizzly]

can't you say the same for transistors representing a neural network? Could that be considered a brain of sorts and thus part of a body? If it cannot be considered the same, what makes it different. Is it because a brain is made from organic material and transistors are not? Or something else? Would like to understand where you are comming from.

[solid_fuel]

I think you're missing his point. It's not about the hormones and physics of our bodies, and indeed he specifically allows for "either physical or virtual" bodies even in the block ~~~you~~~ grandparent quoted.

The point here is not that it must have a body like ours, the point is that a conscious entity must have a boundary line between internal (the body) and external (everything else).

A virtual sense organ can simply be an encoder or a web camera or a magnetometer, the specifics don't matter, what matters is that there are only a few bridges between the outer world and the inner world.

Even if you want to call a tokenizer and autoencoder a "sense organ", LLMs are not embodied because there is no boundary line - there is no internal "thought" that is not directly descended from the prompt and there is no internal reasoning which is not immediately dumped into the external environment.

[cout]

Would it be sufficient to have a second stream of tokens that becomes the model's equivalent of "internal dialogue"? Would that satisfy the requirement of a boundary line?

Related, is a human "thinking out loud" still thinking, even though the internal reasoning is "immediately dumped into the external environment"?

[datsci_est_2015]

> If I'm not experiencing any particular emotion in a given moment, am I concious?

No, you’re not. I think even baseline emotional responses to stimuli is table stakes for “consciousness”.

[rvrs]

>If I'm not experiencing any particular emotion in a given moment, am I concious?

Genuine question: How are you so sure whether you are experiencing an emotion or not? Are you a master of everything inside your own head?

[lwarfield]

>Yeah, it's kind of mind boggling that Ted Chiang (of all people!) can't imagine intelligence without a body. and the whole thing just begs a lot of questions.

Damn, what a line!

Another thing that bothered me with his baseline for consciousness was that it did not involve the ability to change one's self. A big part of being conscious in my mind is how one's experiences shape them, and how someone can shape themselves. LLMs completely lack this, their weights are static. An LLM isn't going to be molded by a bad breakup, or a relative passing away. An LLM isn't going to set up a routine to get stronger with training, nor smarter by reading up on a field.

[sdenton4]

The thing is... Interactive updates happen, just in a different way than it does for animal brains. The system is updated with new training data more-or-less constantly. Suppose OpenAI (or whoever) collects a week's worth of conversations with up-thumbs and down-thumbs, or rewritten continuations from human operators, then fine-tunes the current version of ChatGPT with that data. That's an interactive update, and learning from experience. It looks mostly nothing like what we humans do... But it does rhyme a little bit!

We humans have mostly frozen weights (neurons), or else we would constantly be having to avoid forgetting how to walk+talk. We have a period of greater plasticity (youth!), and use sleep and dreams to perform 'deeper' updates than occur when we're awake: We tend to suck a bit at picking up new skills from zero, but improve rapidly with practice over days.

[andoando]

Yup, you could definitely take it offline (sleep) every night, update it and turn it back on.

[felipeerias]

A stronger version of that argument is that LLMs are not intrinsically affected by the passage of time.

Input stream comes in, input stream comes out. The LLM doesn't care whether this happens once a minute or once a year.

[Muskwalker]

I don't normally experience the time interval between when my input streams shut down every night and reboot every morning either, though.

Nothing prevents one from running an LLM whose harness has a clock and a while loop in it, and it would be weird if its mere lack were really so consequential to consciousness.

[felipeerias]

Exactly, the clock is external to the model. Nothing prevents it from being faster or slower, or even running backwards, because it’s ultimately just another data point in the input stream to a computer function.

Your brain and your whole body exist in time. Even when you are asleep, your body does not flicker out of existence and your brain actually continues working during that time.

[eberkund]

I'm more interested in what a virtual body would entail. To me the root of this idea is around persistent state which is something that currently LLMs do not have. Imagine if somehow your brain lacked long-term memory forming capabilities and instead each day when you woke up you had to read a notebook with (markdown formatted) instructions that you wrote the day before? I wouldn't be surprised if such a person lacked in many of the dimensions we consider important for consciousness, even in less sophisticated forms of life like dogs or mice.

[glaslong]

Yes it seems that is the crux of the embodiment section in the article. That whether physical or virtual, the "AI" needs minimally: persistence in its environment, sensory signals of that environment, and some feedback loop of continuing to try to exist in that environment; having a subjective experience.

And that that is the baseline before we can really even consider that it has consciousness of its own subjective experience, versus being a worm that happens to output text as its digestion process.

And then the further question only after that is established, is what are its needs? What moral patienthood do we have to acknowledge in terms of meeting those needs? And finally, with all the other prerequisites checked, what is the AI's moral agency in what it chooses to do.

[high_priest]

There is no soul in a human. Just a bunch of systems nudging each other to action. What people call soul is literally the same as the concept of personality. In essence, the way all systems in your body have been calibrated to exist.

I believe that the moment an artificial inteligence is going to "receive" a soul, is the moment it is going to be made to sustain itself. Either as a larger package (some bots working to keep an AI farm running) or as an individual (a bot which is tasked with not only fulfilling human desires, but also sustaining itself)

[SethTS]

Are you familiar with Michael Graziano and Attention Schema Theory? I think that is a better "substrate independent" formulation of the objections Ted Chiang is expressing here.

https://en.wikipedia.org/wiki/Attention_schema_theory

[devindotcom]

the very next paragraph addresses this concern imo. it's just an example of one way it might be convincing to him, since of course we are naturally anthropocentric.

[tvshtr]

The same thing jumped at me immediately. He should have prefaced this with his definition of consciousness. Moreover the embodiment of LLMs is already happening via robotics, and virtually. Then there's the common counter "but humans are a next word prediction machines too.." (ofc we're more than this, but linguistically we are, and that's the field from which LLMs originate) which is rarely addressed.

[slashdave]

Indeed.

Hypothetically (and in reality, this is not too far off), if a AI is trained via RL by driving a robotic body, is there a point in time after enough is learned that the AI model becomes "conscious"?

[emtel]

What a strange paragraph! Why on earth should we assume that a body is a prerequisite for having emotions?

[rf15]

Your mind is very much tethered in reality, and your decisionmaking depends on your various inputs, outputs and their consequences. Reducing all inputs to one (text) and all outputs to one (text) is a reduction of all possible ability to perceive and act and thus a reduction of the ability to think.

Multimodal does not change this significantly, considering that nothing is tied to real consequences.

[throwthrowuknow]

The body itself has little effect on the mind other than the inputs from nerves and chemical and hormonal changes. These inputs are analogous to tokens and parameters for an LLM. You could theoretically reserve some tokens to be used as “physical” or “emotional” sensations that would affect the functioning of the model.

[overgard]

That's not really true, the body actually has pretty profound effects on the mind. For instance, there are working theories that many mental disorders are actually metabolic disorder, some neurotransmitters are majority produced in the gut (seratonin), and even things like an elevated heart rate will create emotional states of anxiety even if you're not consciously anxious of anything. That's like the tip of the iceberg. (Also I don't think you can minimize hormones.. ever met a teenager?)

[throwthrowuknow]

Which of these do not involve chemical and neural communication?

[xboxnolifes]

"Other than... chemical... changes" is doing a lot of heavy lifting. The entire existence of your body is a continuous process of chemical reactions.

[throwthrowuknow]

I can narrow that to the chemical signals that can cross the blood-brain barrier if that makes the point clearer.

[gabrieledarrigo]

> The body itself has little effect on the mind other than the inputs from nerves and chemical and hormonal changes.

"Little"...

[goatlover]

There are tens of millions of neurons in the stomach and intestines which do influence mood and motivate behavior.

[throwthrowuknow]

and how do those signals reach the brain?

[arjie]

My claw-like is hooked up to my internal cameras (baby-cams) and to my Dreame Ultra X40. That gives it a body and sense organs since it can check the cameras to see if the living room floor is clear before sending the vacuum off. I don't think that gives it consciousness. Is it the sample rate?

The question is somewhat ill-defined, though. We 'experience' reality continuously because of how we are, but a sleeping human in deep non-REM has the mind not actually active. So they're not a conscious being. So conscious/unconscious is not a line I think easily drawn[0]. Whatever, this stuff is much more well-trodden than this HN comment so I won't rehash. I, too, am surprised that Ted Chiang whose work seems so cleverly novel in so many ways has what seems to me a pedestrian view.

0: very sorites, you know

[plaidfuji]

> The first requirement is that the computer program has a body (either physical or virtual) and sense organs

Ok, deploy a local model on a lightweight edge compute device and strap it to a chassis with wheels, and attach a cheap webcam

> Then I’d want to see an embodied agent that could navigate its environment in order to survive as well as, say, a lizard can

Give the robot appendages that enable it to plug itself into a standard wall outlet, guided by a vision model plugged into its webcam. As long as it can feed itself, it can survive long enough.

> Next I would want to see an embodied agent with the same capacity to deal with novel situations as a mouse.

I think if you fed frames from the webcam into a local VLM every 5s you’d be able to assess a situation and respond with simple actions (turn, advance, retreat).

> After that I’d want to see agents whose social dynamics are as complex as those of wolves, and then agents with the tool-making abilities of chimpanzees.

Social dynamics could be implemented in many ways, maybe by transmitting tokens over RF? Idk. Then you have a scanner that picks them up, feeds them into some LLM frontend and decides whether to add them to a global context file that guides the VLM action-taker. A new action could be to broadcast a token message. Tool-making would have to be code-based. Physical tools are hard. Still unsolved.

> At that point I would want to see people successfully teaching such embodied agents how to communicate their desires

This part is relatively straightforward except for the “via nonlinguistic modality”.

Anyway. These are all engineering problems. Personally I would demand to see the AI reproduce its body under its own power and volition. That’s a pretty neat trick we’ve got going for us.