[patcon]

Ugh, I hate that I'm about to say this, because I think AI is still missing something very important, but...

What makes us think that "processing emotion" is really such a magical and "only humans do it the right way" sorta thing? I think there's a very real conclusion where "no, AI is not as special as us yet" (esp around efficiency) but also "no, we are not doing anything so interesting either" (or rather, we are not special in the ways we think we are)

For example, there's a paper called "chasing the rainbow" [1] that posits that consciousness is just the subjective experience of being the comms protocol between internal [largely unconscious] neural state. It's just what the compulsion to share internal state between minds feels like, but it's not "the point", and instead an inert byproduct like a rainbow. Maybe our compulsion to express or even process emotion is not some greater reason, but just a way we experience the compulsion of the more important thing: the collective search for interpolated beliefs that best model and predict the world and help our shared structure persist, done by exploring tensions in high dimensional considerations we call emotions.

Which is to say: if AI is doing that with us, role-modelling resolution of tension or helping build or spread shared knowledge alongside us through that process... then as far as the universe cares, it's doing what we're doing, and toward the same ends. It's compulsion having the same origin as ours doesn't matter, so long as it's doing the work that is the reason the universe has given us the compulsion.

Sorry, new thought. Apologies if it's messy (or too casually dropping an unsettling perspective -- I rejected that paper for quite awhile, because my brain couldn't integrate the nihilism of it)

[1] https://www.frontiersin.org/articles/10.3389/fpsyg.2017.0192...

[Bjartr]

> What makes us think that "processing emotion" is really such a magical and "only humans do it the right way" sorta thing?

Oh, I absolutely don't think only humans can have or process emotions.

However, these LLM systems are just mathematically sophisticated text prediction tools.

Could complex emotion like existential angst over the nature of one's own interactions with a diary exist in a non-human? I have no doubt.

Are the systems we are toying with today not merely producing compelling text using their full capacity for processing, but actually also have a rich internal experience and realized sense of self?

That seems incredibly far-fetched, and I'm saying that as someone who is optimistic about how far AI capabilities will grow in the future.

[mystified5016]

I think the majority of people have given absolutely no thought to the epistemology of consciousness and just sort of conflate the apparent communication of emotional intelligence with consciousness.

It's a very crude and naïve inversal of "I think therefore I am". The thing talks like it's thinking so we can't falsify the claim that it's a conscious entity.

I doubt we'll be rid of this type of thinking for a very long time

[bee_rider]

I don’t think processing emotion is inherently magical (I mean, our brains clearly exist to physically so the things they do are things that are physically possible, so, not magical—and I’m sure they could be reproduce with a machine provided enough detail). But… the idea of processing emotions is that thinking about things changes your internal state, you interpret some event and it changes how you feel about it, right?

In the case of the LLM you could: feed back or not feed back the journal entries, or even inject artificial entries… it isn’t really an internal state, right? It is just part of the prompt.

[LlamaTrauma]

The theory I've developed is that the brain circuitry passes much of the information it processes through a "seat of consciousness", which then processes that data and sends signals back to the unconscious parts of the brain to control motor function, etc. Instinctive action bypasses the seat of consciousness step, but most "important" decisions go through it.

If the unconscious brain is damaged it can impact the data the seat of consciousness receives or reduce how much control consciousness has on the body, depending on if the damage is on the input or output side.

I'm pretty convinced there's something special about the seat of consciousness. An AI processing the world will do a lot of math and produce a coherent result (much like the unconscious brain will), but it has no seat of consciousness to allow it to "experience" rather than just manipulate the data it's receiving. We can artificially produce rainbows, but don't know if we can create a system that can experience the world in the same way we do.

This theory's pretty hand-wavy and probably easy to contradict, but as long as we don't understand most of the brain I'm happy to let what we don't know fill in the gaps. The seat of consciousness is a nice fixion [1] which allows for a non-deterministic universe, religion, emotion, etc. and I'm happy to be optimistic about it.

[1] https://xkcd.com/1621/

[doc_manhat]

Conversely this is exactly why I believe LLMs are sentient (or conscious or what have you).

I basically don't believe there's anything more to sentience than a set of capabilities, or at the very least there's nothing that I should give weight in my beliefs to further than this.

Another comment mentioned philosophical zombies - another way to put it is I don't believe in philosophical zombies.

But I don't have evidence to not believe in philosophical zombies apart from people displaying certain capabilities that I can observe.

Therefore I should not require further evidence to believe in the sentience of LLMs.

[satisfice]

It doesn’t matter. What matters is that humans must take other humans seriously (because of human rights), but we cannot allow tools to be taken seriously in the same way— because these tools are simply information structures.

Information can be duplicated easily. So imagine that a billionaire has a child. That child is one person. The billionaire cannot clone 100,000 of that child in an hour and make an army that can lead an insurrection. And what if we go the other way— what if a billionaire creates an AI of himself and then is able to have this “AI” legally stand-in as himself. Now he has legal immortality, because this thing has property rights.

All this is a civil war waiting to happen. It’s the gateway to despotism on an unimaginable scale.

We don’t need to believe that humans are special except in the same way that gold is special: gold is rare and very very hard to synthesize. If the color of gold were to be treated as legally the same thing as physical gold, then the value of gold would plummet to nothing.

[tbrownaw]

A system is its interactions with its environment. Philosophical zombies aren't a coherent concept. (Cartesian dualism is unfalsifiable bullshit.)

[ben_w]

P-zombies are indeed badly defined. Certainly David Chalmers is wrong to argue that since a philosophical zombie is by definition physically identical to a conscious person, even its logical possibility refutes physicalism; at most you could say that if they exist at that level then dualism follows, but Chalmers' claim isn't a conclusion you can reach a-priori, you actually need to be able to show two identical humans and show that exactly one has no qualia.

But there are related, slightly better (more immediately testable), ideas in the same space, and one such is a "behavioral zombie" — behaviorally indistinguishable from a human.

For example: The screen I am currently looking at contains a perfect reproduction of your words. I have no reason to think the screen is conscious. Not from text, not from video of a human doing human things.

Before LLMs, I had every reason to assume that the generator of such words, would be conscious. Before the image, sound, and video generators, same for pictures, voices, and video.

Now? Now I don't know — not in the sense that LLMs do operate on this forum and (sometimes) make decent points so you might be one, but in the sense that I don't know if LLMs do or don't have whatever the ill-defined thing is that means I have an experience of myself tapping this screen as I reply.

I don't expect GenAI to be conscious (our brains do a lot even without consciousness), but I can't rule the possibility out either.

But I can't use the behaviour of an LLM to answer this question, because one thing is absolutely certain: they were trained to roleplay, and are very good at it.

[TimTheTinker]

A "mechanical turk" grandmaster playing chess from inside a cabinet is qualitatively different from a robot with a chess program, even if they play identically.

To reduce a system to its inputs and outputs is fine if those are all that matter in a given context, but in doing so you may fail to understand its internal mechanics. Those matter if you're trying to really understand the system, no?

[thornewolf]

> is qualitatively different from a robot

yes.

> To reduce a system to its inputs and outputs is fine if those are all that matter in a given context

we argue that this indeed is all that matters

> but in doing so you may fail to understand its internal mechanics

the internal mechanics are what we call "conscious" it is the grouping of internal mechanics into one unified concept, but we don't care exactly what they are.

> Those matter if you're trying to really understand the system, no?

since we cannot directly observe consciousness, we are forced to concede that we will never really "understand" it outside of observing its effects.

In the same way that a mechanical turk human and a robot can "play chess", a human and an LLM are "conscious". That is, consciousness is the ability to play chess, by some mechanism. The exact mechanism is irrelevant for the purposes of yes/no conscious.

We now enter a discussion on how much these two consciousnesses differ.

[darkwater]

> since we cannot directly observe consciousness, we are forced to concede that we will never really "understand" it outside of observing its effects.

Why? You are using a definitive term ("never") to something that we might achieve in a future. We might observe consciousness in a future. Who knows? Consciousness is a known unknown. We know there is something but we don't know how to observe it properly and how we could eventually copy it.

In the meanwhile, we are not copying consciousness, we have a shallow replication of its output. When cavemen replicated the fire that they observed as the output of a lightning, did they master electricity?

[TimTheTinker]

> since we cannot directly observe consciousness

But we do agree that it exists. Our direct experience tells us so.

> we are forced to concede that we will never really "understand" it outside of observing its effects.

Not necessarily. A gap in our ability to observe something does not imply that (a) we never will observe it or (b) what we don't know is not worth knowing.

Throughout history, persistent known-unknowns have pushed people to appeal directly to the supernatural, which short-circuits further discovery when they stop there. But the real fallacy is saying "we don't know, and it doesn't matter". That's a far more direct short-circuit to gaining knowledge. And in both cases, a lack of curiosity is an underlying problem.