[h0l0cube]

> Currently, the company is working to get its mini-brains—which so far are approaching the processing power of a dragonfly brain—to play the old Atari arcade game Pong

I'm surprised that no-one has discussed whether these systems could develop emergent qualia, and experience pain. No joke. Are there any ethical frameworks around this kind of research?

[nnq]

> could develop emergent qualia

If we were to admit there really is such thing as "qualia", there's not reason you wouldn't ask the same question about non-biological software or hardware systems... electronic hardware or biological hardware can have same computational qualities.

But since OBJECTIVELY there's no such things as qualia (the concept only exist SUBJECTIVELY), it could only exist by definition for "a person/subject" eg., in this context, a neural network large and complex enough to get close to a human-like intelligence level!

A dragonfly nervous system is probably simpler than some of the the largest artificial neural networks models...

[MaxBarraclough]

I'm inclined to agree. To put it another way, then: what matters isn't the substrate (neurons vs transistors), but what you do with them.

Going this route, we'd have to say that if you wrote a program that perfectly simulated the human brain, you've built a conscious system running on transistors.

It also means that if you built a RISC-V processor based on a large number of neurons (say, as many as are in a human brain), you haven't built a conscious system, despite that it's neuron-based.

I'm ignoring the obvious problems with building a RISC-V using neurons; they strike me as incidental detail, for our purposes here.

[h0l0cube]

I've answered to these points in another comment. But I'll emphasize, that I know (and presume you know) within some minimum epistemic bound^, that we can suffer pain and that arises from our neurological systems. It can't be known what other systems/substrates could experience qualia, and even if they have the outward markers of consciousness, they could be 'zombies'.

https://en.wikipedia.org/wiki/Philosophical_zombie

^ i.e., not in some adversarial simulation etc.

[eyeundersand]

> A dragonfly nervous system is probably simpler than some of the largest artificial neural networks models.

I would be extremely surprised if that were the case. Behavior of a single cell alone is unimaginably complex. Let alone an organ system, let alone the nervous system.

I will agree with your Qualia point.

[colordrops]

Are only biological systems capable of experiencing qualia? What is to say that a silicon computer can't also have subjective experiences?

[h0l0cube]

I do sometimes entertain the idea of panpsychism (like perhaps a rock is conscious), and consciousness may not be substrate dependent, but I can be certain, by at least my own anecdata, that neurons give rise to qualia.

The jury is still out on this, but a big-world network with some quantum computation could be enough. The 'hard problem of consciousness' is that we can't really know by observing a system whether it is conscious or not, or rather a sort of 'zombie' that has all the features resembling a conscious being without qualia.

[Rexxar]

Arguably, computers seems to suffer when there are be computations : they becomes hot and start to breathe a lot. That's a little irrational be I used to feel a little bad when I saw my old computer "suffering" when I was younger.

[MaxBarraclough]

Seems unlikely that's suffering, any more than petrol suffers.

Animals (like us) suffer as part of our evolved aversion response. We have an in-built capacity for preference. Current-day computers lack this.

[spyder]

Just because you are hot and breath a lot it doesn't mean you are suffering, you are just working more. You start to suffer when you work over your limits, like when you overclock a computer too much and starts to glitch because some parts of it cannot handle it.

[collyw]

That's the big question that science so far has no clue about.

[tartoran]

I am not an expert but from my understanding pain are signals perceived as what we subjectively call painful when something in our body/nervous system is damaged/inflamed or a system is out of whack. It is a very useful thing, it is trying to tell us something useful so we can act on it or just be aware of it. When something is out of whack permanently something in this system is malfunctioning. Without a body it is hard to describe the concept of pain and what it may feel to an emergent system with qualia properties. The closest I could think of is some kind of distress signal or base frequency change when the said system is under stress or wrong state and is perhaps used to self correct the system to a balanced state. If something is permanently painful then it ceases to be a reliable signal.

Psychological pain or emotional pain or distress may come closer to what such a system could experience and but it could also be an emergent property of such a system but we don't fully understand how this works. If such a system has emotional capability then it should be able to experience it because pain is such a state of the emotional spectrum. No pain no joy - no oscillation/flat

[totalZero]

We are all machines. The idea that one type of machine or another cannot feel "pain" is simply a rationalization for human behavior that causes harm to other organisms, like hunting and fishing and killing wasps.

Pain is a sensation that drives an animal to quickly react to injurious situations. Consider a robot made with Brooks's subsumption architecture [1]. The basal level of behavior drive of the robot is its self-preservation. That level of behavior drive can halt all the higher-order behaviors to monopolize processing and locomotion. Imagine that this behavior drive is triggered when the ambient temperature approaches a level that can damage the battery, and attempts to move the robot to the lowest-temperature zone available. That is functionally no different than what happens to a person in a house fire.

All we have done with the word "pain" is to set up an arbitrary delineation based on the hormonal/neuronal response that links our sensory apparatus to our behavioral drive.

[1] https://people.csail.mit.edu/brooks/papers/representation.pd...

[ben_w]

Careful, or you might use a definition of pain too broad to be useful.

Plants, single-celled organisms, thermostats, and anti-lock braking systems meet your use of the word.

Almost all of us know what it means to be in pain. It’s the hard to put into words subjective experience of suffering that is worth making ethical decisions about, not the autonomous reflex that doesn’t come with conciousness-is-a-poor-word-but-I-lack-a-better-alternative.

[totalZero]

I disagree with the presented interpretation of pain as a concept. That's not the same as lacking an alternative word. Be careful not to make this discussion a semantic one.

I have no reason to believe that what a Roomba feels when it approaches a descending flight of stairs is any different in practice from what I feel when I approach a precipice. Our pathways are totally different, but ultimately there is a communication from our sensory organs to our processing architecture, where an ingrained drive toward self-preservation momentarily overrides other needs. In my case, I could call it "fear," or "anxiety," or "angst" if I'm feeling philosophical. In the Roomba's case, I have no knowledge of its subjective experience, but that doesn't mean that I should draw up a new word to construct a delineation between the Roomba and myself. We are both machines, and our response to the same situation is largely the same.

How useful is it to describe pain as a "subjective experience of suffering" when neither subjective experience, nor the feeling of suffering, is directly observable? (Also, I don't agree that pain and suffering are interchangeable concepts.)

A truly useful definition of "pain" would hold water without reliance on an anthropocentric tautology. I know what it feels like to slice my hand open while cutting tomatoes. That doesn't give me any power to understand what it feels like for an octopus to lose one of its arms, or for a tree to have its branches trimmed.

What's funny is that my way of considering "pain" is not even the most divergent from yours. Many Andean cultures believed that stones, rivers, and mountains have energy, thoughts, feelings, and souls. These ideas remain in the culture to this day.

What spurs an ABS to pulse the brake line pressure when the wheels begin to slip as the driver mashes on the brakes to avoid a collision? What spurs an ant to run away when it steps onto a hot radiator? What spurs a human to stop walking on a broken ankle? All of this is programmed in one way or another, all of it is self-preservation.

[ben_w]

All of our emotions are indeed programmed. Yet the humour axis does not feel like the pain axis, neither feels like the sexual arousal axis, and none of those feels like the fear axis — at least not to me.

Also, my knee-jerk reflex doesn’t feel like much at all.

If I read you correctly, you and I agree that we can’t tell if a Roomba’s avoidance algorithm more like anxiety, or more like a reflex, or more like lust. In the absence of evidence, I will assume that all its experiences are like a reflex, that there isn’t anything “that it’s like to be” a Roomba. I am aware I may be wrong.

While we indeed cannot directly observe subjective experiences, I sincerely hope that we figure out a good way to resolve this soon:

If we mistakenly assume AI cannot have qualia, then we risk condemning our creations to torment from which they are only released by their own destruction.

If we mistakenly assume we can create machines which have qualia, then brain uploads are death.

[tartoran]

I agree with you, pain is a useful signal or it could be a systemic damage that wrongly sends these signals. However, when we, humans, refer to pain we clump all the unpleasant things we experience together. One such pain is the emotional /psychological pain and this type of pain is more likely to occur in such a system without a body. This is also a signal/state and is also useful to delineate it from another state. If such system is to experience joy it better know what pain is or otherwise wouldn't be able to delineate.

[h0l0cube]

Are you a robot?

If you are at all in doubt of your qualia, to quote Sam Harris (who has many entertaining podcasts on this topic with people far more qualified to speak on it than I am):

> Unfortunately, many experiences suck. And they don’t just suck as a matter of cultural convention or personal bias—they really and truly suck. (If you doubt this, place your hand on a hot stove and report back.)

https://twitter.com/SamHarrisOrg/status/951276362387591169

The twitter thread was about moral realism, but the topics are very much intertwined. If there is no imperative for anyone to do less harm to other creatures, why should anyone care if you are in pain?

And of course this argument extends to our treatment of animals too. Countless number of living, feeling animals, suffer for scientific progress (and cosmetics), let alone factory farming and wet markets. But at least there's light at the end of the tunnel for some of these concerns (lab meat, improved in-vivo testing)

[totalZero]

Do you believe that a robot cannot have qualia?

I am a collection of interconnected units, each of which is itself a combination of different organelles and other purpose-serving features. My intelligence is itself a product of the interaction of simple, unintelligent parts. So I would say that I am a machine. I would describe a robot as a man-made machine where the interaction of various silicon parts and processors gives rise to intelligence. By that definition, I am not a robot.

I believe that we don't feel pain when we are truly in trauma. It is only when we can do something about the damaging stimulus that we feel pain. I once had a major accident where I experienced lung collapse and multiple fractures. I never lost consciousness but I don't remember feeling any pain until well after others came to my aid. Even when I tried and failed to pick myself up off the ground, I did not feel pain, only disability and relief that my fingers and toes still moved.

I believe that the question of "what is pain" and the question of "is pain a good criterion for deciding whether it is acceptable to do harm to something" are two totally different philosophical problems. They are connected only insofar as we have chosen pain as a proxy for harm. But that very relationship between pain and harm indicates that pain is not just some kind of soulful feeling, but rather a signal to help us evade harm now or in the future.

[h0l0cube]

> Do you believe that a robot cannot have qualia?

We're getting stuck on semantics here. I do, but then I'd cease to see it as a robot, and more a sentient being. One criterion of consciousness that I've encountered is, 'there is something what it is like to be <x>' (Thomas Nagel). If there's something 'what it is like to be' a robot, a bat, a mosquito, an ameoba, a rock... then it is conscious.

> I believe that the question of "what is pain" and the question of "is pain a good criterion for deciding whether it is acceptable to do harm to something" are two totally different philosophical problems. They are connected only insofar as we have chosen pain as a proxy for harm. But that very relationship between pain and harm indicates that pain is not just some kind of soulful feeling, but rather a signal to help us evade harm now or in the future.

Qualia is more broad than just pain, of course. I just picked this particular phenomenon for it's poignance :)

If it is conscious, ethically speaking, we should consider how we treat it in a manner different to something that isn't. So if a rock isn't conscious, and some interconnected neural/silicon device is, we should at least have some way to query whether it is in an undesirable state or not.. if feasible/practical.

Maybe if trees/plants/rocks/ameobas are conscious, we can't be consulting their feelings when we harvest crops, or use disinfectant, mine for precious metals etc. We can make decisions to treat livestock better, and change how we utilize our environmental resources - so we ought to, and we are. But if we were to go out of our way to make new conscious entities, don't you think we should extend our historical shifts in attitude to slavery and our growing shifts in attitude to animal welfare also to these new entities?

[skosch]

The one quibble is that computationalism – the idea that experience is simply what some kinds of computation feel like from the inside, regardless of the substrate – may or may not be correct. It could be that qualia can only arise in systems that are physically intertwined in particular configurations (see Tononi's IIT), and it could even be that quantum effects are required (I'm skeptical, but who knows). The jury is still out on those question.

Therefore, it may be true that using biological neurons, arranged in a certain configuration, would give rise to qualia like pain in a way that shifting electrons between CPU registers never could. We just don't know.

For more reading, here's a relevant whitepaper by the Qualia Research Institute: https://opentheory.net/PrincipiaQualia.pdf

[Der_Einzige]

Hahaha, no. There is no ghost in the Shell in AI powered robots and certainly not in inanimate objects. You sound like an object oriented ontology writer with this kind of post and will be treated as such because that's what you've advocated for.

The alternative is fear of hurting objects or machines everytime I act in the world. Physical objects themselves are not ethical actors. AI is not life and we are several hundred years away from being capable of creating life ourselves within an object (if it's at all possible) Your worldview would mean that if I move a machine (or rock) and it appears to "resist me", I must fear that the machine or rock doesn't want to be moved. I must believe that I have acted violently towards that object by "frustrating it's preferences".

What absurdity!

[totalZero]

Why are you linking pain and ethics? If I chop down a tree in the wintertime, I can reflect upon whether it felt pain from my chainsaw as I drag its trunk toward my house, happy to fuel my family's fireplace with its remains. If it felt pain, does that make my action less ethical than if it did not?

Is it wrong to slaughter a cow because that cow will feel pain?

Sometimes, people label ideas for the purpose of sorting and better considering them. Other times, people label ideas for the purpose of compartmentalizing them and rejecting them without really reflecting on them at all. I don't know what you mean when you say I will be "treated as such" but it sounds like a way of setting aside what I actually said in order to tussle with a preconceived, crystallized notion of something someone else said or wrote.

There are ancient cultures that believed it was wrong to do violence against a rock. Do you believe they, too, read Heidigger?

[taneq]

I don't think qualia are real in the general objective case, only in the subjective case. So nobody but 'you', whoever you are, can actually have them.

[MaxBarraclough]

Meaning what? That doesn't bring any light to the question of whether these things are capable of suffering. As someone else said, this is a hard problem to reason about.

[taneq]

In this case, the complexity is in how you define the word 'suffering'. Is a worm suffering when it tries to escape from a heat source? Is a human infant suffering when it cries in response to a needle prick? Is a motor driver suffering when it raises a high-temperature alarm? As far as I can see they're all on the same continuum, and it's possible to find people who will answer either way to any of the above.

[altivasta]

I think they can only have conscious thought if you perceive something having it. How could it exist without observation if the realm of existence exists within you, who is experiencing the the truth?

Does a scribble on a stone contain something any other formation of cracks wouldn't? No, the meaning is generated within _you_, and as such sentience can only be summed as only your own process observing phenomena

[CaptArmchair]

This is the point where you end up in the realm of philosophy and phenomenology.

[MaxBarraclough]

Correct. The question stands.

[CaptArmchair]

The experience of reality happens on an individual level. It sounds self evident. The corollary is that the feelings you feel are yours and yours alone to feel.

When we use language to describe our experience - pain, joy, loss, fear - we don't transfer the experience proper. What we share is an expression of our experience.

If you describe your current experience as "joy", then you arrive at that expression because - arguably - you learned that concept from your parents as a toddler. I say "arguably" because this is where the "nature vs nurture" debate kicks in. I'm not going to delve into that can of worms.

Suffice to say that we can't transfer our exact experience to another individual; but that we can express our experience and that others are able to interpret that expression. And that a shared understanding comes from a shared conceptual frame of reference.

Art is a great way to challenge that shared frame of reference. Understanding art implies trying to interpret what an artist is expressing. Sometimes the artist doesn't intend to express their own experience, but rather evoke a particular experience or even contradicting experiences within the audience. That's why some art throws us from our feet.

But this isn't about art. This is about suffering. That's where concepts such as empathy and compassion come into play. Those concepts refer not to the actual experience of suffering proper within individuals, but rather the capacity to recognize the experience of suffering in others through their expressions. If you see someone express grief through crying, apathy, irritation,... then empathy means that you understand that they are experiencing grief and you are able to also have that same experience behind the expression as an individual, independent of what others are feeling.

Empathy, compassion and a shared understanding work best if there's a close resemblance between you and the other. Hence why there's little doubt about the experiences of family, close friends and so on. It starts to become harder when you think about suffering in the context of people with different cultures and languages whom you've never met and who live on the other side of the globe. Animals? There's another level that increases that distance. Sure, your pet may be expressing their experience, but how do you know you're not projecting?

And so, here we are, considering neural networks and neurons attached to breadboards. Do they have a similar experience of reality? Do they experiencing suffering in a way that matches with what humans conceptually and "broadly" understand as "suffering"? It's not like we can ask, and even if we could, how would we possibly be able to interpret the expression of "suffering" they convey to us?

For instance, up until the 1970's, infants up until 15 months simply didn't receive anesthesia for surgeries. Everyone assumed that they didn't feel pain because the shared frame of reference among medical practitioners didn't allow for the interpretation of their expression as "oh, they are in pain".

https://www.nytimes.com/1987/12/17/opinion/l-why-infant-surg...

And so, one argument could be that we are already torturing neural networks and we aren't even aware we're doing it because we simply lack a common frame of reference to pick up the expressions of an experience of pain. Hence the cautious reluctance to sanction free experimentation that unwittingly may elicit the experience of suffering.

[prox]

Its called Theory of Mind https://en.wikipedia.org/wiki/Theory_of_mind

[s_gourichon]

Ok let's rephrase then: can those mini-brains be a "you"?

[MaxBarraclough]

Mandatory link to Wikipedia's article on Nagel's What Is It Like to Be a Bat? [0]

> Nagel famously asserts that “an organism has conscious mental states if and only if there is something that it is like to be that organism—something it is like for the organism."

[0] https://en.wikipedia.org/wiki/What_Is_It_Like_to_Be_a_Bat%3F

[taneq]

Of course... but so can any appropriately constructed control system, and so can any living organism which senses its surroundings to some degree.

[dr1337]

I guess if the stem-cells are taken from you as a donor then they would be genetically identical to you as much as twins are genetically identical individuals?

[russdill]

We're already really good at building mini-brains that can play pong..and much more advanced games than pong. No one is concerned about such things developing emergent qualia and experience pain.

[Sosh101]

I think that is a very valid concern, but how could this ever be assessed

[anonytrary]

If something cannot be assessed, then are we still accountable for the result? Or, do we simply discard the assessment?

[ganzuul]

Human neural networks definitely are capable of suffering. These researchers might be re-inventing slavery.

[meowface]

So could any future AI researcher, regardless of the substrate.

I suspect this idea is so widespread at this point that these researchers and others are aware of this possibility and see the potential moral issues. I just think they believe such a thing is still decades (or longer) away, and I think they're probably right.

Right now, just about everyone wildly sensationalizes things that appear to suggest conscious man-made entities, like the AIs that supposedly "invented their own language". I think we're so eager to see it even when there's nothing close to it that once it actually does happen, we'll realize it quickly and understand the implications.

There is the possibility that it could somehow develop despite no external signs of it, but all we can do is try to be as cognizant of the possibility as we can.

[battery_cowboy]

Read "The Quantum Thief" by Hannu Rajaniemi

[collyw]

They could give the artificial brains a bank account and pay them a salary.

[austinjp]

Or discard the project, since the key variable will remain forever unknowable.