[eszed]

Yeah, I currently suspect that consciousness is an emergent property. I read elsewhere (it's somewhere in my HN history, I'm sure) that the biggest compute we can currently muster is something like three or four magnitudes away from the number of neurons / connections (or their analog) that our brains have, so it may be a while until we can expect to see it in our machines. But, if the emergent phenomenon hypothesis is correct, then we eventually will. I'm more scared than pleased by the prospect, but there you are.

[ProllyInfamous]

>consciousness is an emergent property

You would really like Michael Pollan's latest book [1], entirely devoted to his exploration of consciousness researchers' POVs on this exact topic.

My favorite quote is that ~"perhaps Descartes was only half-wrong when suggesting I think, therefore I am; it seems rather closer to I FEEL, therefore I am."~

[1] <https://www.amazon.com/World-Appears-Journey-into-Consciousn...>

----

I've grown thousands of plants; I've read two of the author's other books devoted to plants; in this book Pollan makes compelling arguments for plant sentience (over a much-longer timeframe).

Sure, perhaps plant consciousness is a bit of a stretch, but they're certainly intelligent and curious creatures. He makes both arguments supporting plant volition.

----

If you haven't seen My Octopus Teacher (Netflix), do. I'm a bald 275lb bluecollarguy... and I wept/awed (both). So beautiful, we bundled neurons.

----

Bonus quote ~"color is where reality and magic appear as-if together"~ [color isn't real, but is perceived]. We most-often see what's most-predictable, not necessarily what we actually detected [in the case of color: nothing but nanometers].

[bulbar]

Just to be sure: The "neurons" in today's AI have nothing to do whatsoever with real neurons.

What we can do is simulate very simple brains by simulating relatively few neurons as they appear in worms. In this sense we are multiple magnitudes away where the increasing complexity implies exponential increasing difficulty.

I would think we are so far away that there will be unknown unknowns we encounter on the way.

[vkazanov]

Yes, physically absolutely nothing. But conceptually they seem to to form this very generic function from inputs to outputs that neurons also form.

[bux93]

Only if you ignore almost every input and output that neurons have.

https://www.quantamagazine.org/ai-is-nothing-like-a-brain-an... https://pmc.ncbi.nlm.nih.gov/articles/PMC9665914/

This is why making more neuromorphic NNs is still an active area of research, although they typically all focus on another extremely simplified model (spiking neural networks).

[vkazanov]

I don't ignore anything. I just refuse to accept the magical thinking around biological machines that are our brains/bodies. There are inputs, there are outputs, there is hidden function.

And it seems that, given enough input/outputs/compute, it is possible to train the necessary function.

Details of how the building bricks look like (matmul, electromagnetism or quantum effects) are not that relevant in the broader picture.

What is missing right now, is the fact that the function in question changes over time in biomachines, while our LLMs are static at inference time.

[davrosthedalek]

I mostly agree, but I see two points that might be problematic:

a) The brain might have an entropy source (then it can't be modeled as a function). Trivially to fix, and in some sense, with diffusion models starting from random numbers, AI has done so.

b) The hidden function might be not computable. I would have no idea how that would work, but I think this is what it boils down to if people say "the human brain is more than a machine".

[vkazanov]

a) enthropy can be injected as well. In fact there are hidden sources in current training.

b) well, it can be the case that, say, certain kinds of computation are either too inefficient or outright impossible within the current model.

Who knows...

[fc417fc802]

Agree and add, don't confuse the substrate for the computation. Of course it's also clear that we don't quite have a full and definite picture of what the computation consists of in the case of a biological brain as evidenced by our continued failure to accurately simulate even the simplest of organisms.

[lelanthran]

> Just to be sure: The "neurons" in today's AI have nothing to do whatsoever with real neurons.

Yeah, but it's hard to explain this to people, especially AI-pro people. Too many are convinced that all we are doing is a cut-down version of the human brain, and it's hard to explain to them that, no actually, we aren't modeling the human brain to the level of granularity you think we are.

[ffwd]

Personally I'm not a fan of the emergence story, for a number of reasons. First is, it doesn't really make sense for consciousness to have emerged gradually through natural selection. When we and animals are conscious, the whole brain is coordinated and works to for example turn off consciousness when we sleep. And as someone else mentioned, animals with much fewer numbers seem to have a similar consciousness to us.

If consciousness really evolved gradually, you would expect to see for example dogs or gorillas having less of it, but if they has less of it, why does it function the same way? Like for example animals can be scared, happy, anxious etc, they can experience the full range of emotions and thoughts, so their conscious experience seems just as rich as ours. What I mean by this is, if you can be "less conscious", then what does that mean _exactly_? Is it that you have less content in consciousness, or is it that you feel more like you are asleep? Or something else? We don't have any examples in animals of "less conscious", I would argue.

This makes me think that rather than having emerged gradually, evolution found a mechanism by which consciousness exists, and then some animals have that mechanism and others don't. I think that if it is a mechanism, then this mechanism is located in one part of the brain, not many parts functioning together (though one possibility is that this mechanism coordinates brain activity in such a way to enable consciousness).

[eszed]

I proposed in another comment that consciousness and self-awareness are at least close cousins, and perhaps the same phenomenon. If that's true, then that's an axis upon which you might create comparative measures. Yes, hamsters are conscious, but they don't have a sense of self to the same degree that gorillas do. If you posit capacity for language as another emergent property of sufficiently-complex networks, then you have another measure.

LLMs, then, are particularly unintuitive to us, because they've got to the language part first, long before they've reached even hamster-level self-awareness. They're not, however, biological networks, so there's no reason these properties need arise in the same order, or indeed in the same ways.

I'm not entirely convinced by that second paragraph, but I think the logic holds together.

[ffwd]

I'm not sure consciousness and self-awareness are the same thing. First is we can be conscious when we sleep/during REM sleep, where it's arguable we are not self-aware. And if not that, we can even do it when awake, for example when we think about a movie, or a philosophical problem, we can have conscious thoughts that are not related to the self. This leads me to believe consciousness is separate from self-awareness. Self-awareness is _one thing_, among many, that the brain can think about and be conscious of.

[eszed]

Sure, but capacity for self-awareness? I'd guess that hamsters dream, and that their subconscious processes (eg, desires for food and sleep and sex), and maybe even emotions, run much like ours. It's just that humans, with more complex neural networks, have more layers added on top. It's similar to how the brain-stems of everything from lizards on "up" function similarly, but humans have more-developed pre-frontal cortexes and so forth. (Don't hold me to those details, please, I'm not a neuro-anatomist! You can see where I'm going with that, though, right?)

[ffwd]

Well in that case I'm not sure where you're going. I agree that hamsters probably have a similar consciousness to ours, which is kind of the point I was trying to make.

I think that consciousness comes before self-awareness, even though self-awareness is kind of a vague term. Self-awareness can either be an abstract knowledge that you are an organism and a discrete entity in the world (world knowledge/self knowledge), or it can be more basic and be a form of conscious experience, but as my point was, I think conscious experience is broader and does not necessarily need to be about self-awareness.

[eszed]

I think we agree on all of those points. I'd posit, however, that whatever "it" is, and however you define it, humans are further up the scale than other animals. That's partly why I mentioned language use; it could be the step function that allows further development. Like, sophisticated-enough communication with other conscious entities is the factor that unleashes recursive improvement. This is all highly speculative.

[kevin_thibedeau]

Our machines won't have biological systems driving their needs which in turn fuel behaviors like desire and planning for the future. They may imitate them but it won't be innate.

[trick-or-treat]

For an LLM, "innate" means "in their training data". So yeah, those things are pretty much innate.

[ludwik]

And also "instilled during their reinforcement training", and we are currently pushing planning hard there, for autonomous agents.

[trick-or-treat]

No I think reinforcement training would be an example of not innate. Don't you? That's like potty training.

[ludwik]

Is it? Both supervised learning and reinforcement learning are ways of training the model, and the difference between them is not that big. I would say that innate means "in the weights", while non-innate means things the model learned during inference, during its "lifetime".

[trick-or-treat]

Maybe you're right. In the weights might be the right way to frame that. What do you mean by "during its lifetime"? Do you mean things like system prompts or things in Claude.md?

It sounds like you're framing a session as a "lifetime". Whch might be right, I haven't thought of it like that before though. So when I /compact my session what's that even the equivalent of I wonder.

[psychoslave]

When we say "it rains", do we consider that "it" has any intention of agentivity?

Some questions are just ill formed.

Plus even if "LLMs are alive and conscious", this still would scratch the surface of the morale/ethical/societal considerations that people really care about.

Because even with other humans, we can argue if they exist or if they are mere npc in a solipsist world view.

[trick-or-treat]

I don't think we say "it rains", do we?

We say "it's raining" but that doesn't imply agency to whatever is causing it to rain.

Also, please don't invoke solipsism, if you want to debate that, you're by definition obligated to do it with yourself.

[psychoslave]

Good catch. For my defense not an English native speaker here, plus apparently it’s not completely agrammatical¹, though I’ll admit indeed "it’s raining" would have been a far more idiomatic example. Thanks for pointing.

About solipsism, yes sure.

I didn’t had time to add it at the moment, but I initially also wanted to discuss more on the prosaic points. So here we go:

Recognizing sentience/consciousness to something actually is not that a big factor in defining behavioral response. Let’s just look at how humans are treating other humans, other mammals, other animals, other life forms. Or taken the other cultural scheme around, in some animist folk one can perfectly consider that any rock on the ground have really an inner soul life just as much as oneself, it doesn’t mean it will consider possible interactions with the rock as equivalent. And, wink to the Overton window, practicing institutional cannibalism doesn’t imply that eaten people are disrespected in their dignity, quite the contrary.

All that to say that "are LLMs conscious" is not even a moral important point. Paperclip maximizer² and can the famous Dijksta’s quote "The question whether computers can think is not more interesting than the question whether submarines can swim." already show how pointless morale debate on LLM and consciousness are, even when one consider potential logistical threats/opportunities they could represent.

¹ https://english.stackexchange.com/questions/123739/its-raini...

² https://www.lesswrong.com/w/squiggle-maximizer-formerly-pape...

[lotyrin]

I think those are things human consciousness has, not is.

[clort]

Surely the number of neurons is not the whole story. Human brains have approximately 86 billion and are almost definitely conscious, but there are many other animals with a lot fewer (gorilla: 34 billion, dog: 2-3 billion, guinea pig: 240 million) which appear to be conscious also.

https://en.wikipedia.org/wiki/List_of_animals_by_number_of_n...

[slopinthebag]

This is not meant as a gotcha, I am genuinely curious how you believe consciousness can be an emergent property. I assume you don't believe consciousness is a physical property in the brain, so what entity is actually experiencing that consciousness? Or, what does it even mean to experience consciousness? Or are these not even the right questions?

[doctoboggan]

> This is not meant as a gotcha, I am genuinely curious how you believe consciousness can be an emergent property.

I was about to post the exact opposite question? How could it not be an emergent property? Unlike consciousness, the concept of emergence is pretty well defined: An emergent property is a characteristic or behavior that a complex system has, but which its individual components do not have on their own.

Consciousness itself doesn't have a well agreed upon definition, but I would posit that _most_ people would agree humans have it, and _most_ people would agree individual cells (neurons) do not have it. If you agree with those two statements, then consciousness is an emergent property by the definition I gave above

[throwaway173738]

I think consciousness is going to turn out to be very challenging to define rigorously enough that we can test for its presence or absence. Emergent or not, the question is how do you determine when it has emerged? Is it a quantity or an attribute? Discrete or continuous? Does it have a finite or infinite range?

We can all agree on what color something is, but we can’t describe the color a priori, only by example. I think consciousness may be a similar phenomenon and the only test is by shared experience. If so then we are in deep trouble because we will not be able to anticipate when a system becomes conscious.

[darkwater]

I like the color example, but colors can be reproduced at will by trial and error and by observing the results (and collectively compare them to our shared experience).

Why cannot this be applied to consciousness as well? I mean, it's surely much more difficult to do compared to colors but... impossible?

[adaml_623]

You and previous comment seem to agree that trial and error/shared experience can determine if consciousness has emerged. And this might/will be challenging.

Previous comment used the word "anticipate" and I think they mean that we won't know in advance before we run the trial and error process.

When they say "deep trouble" I assume they mean because creating a non-friendly conscious AI might pose an existential risk for humanity.

However there is also the ethical issue of creating a consciousness and then destroying/murdering it.

[throwaway173738]

Basically all of that, yes.

[bulbar]

I think the alternative is that our brain, somehow, is connected to some metaphysical aspect of reality which is what most religions believe.

[InsideOutSanta]

The counter to that is that altering the brain directly alters the consciousness. I can take LSD and I literally change. I can have parts of my brain removed and parts of my self disappear. It's not like cutting off a leg, where I lose capabilities but am still the same me.

The logical conclusion is that the brain makes me.

[mahogany]

> The logical conclusion is that the brain makes me

The logical conclusion under the popular axiomatic framework of materialism, for sure.

But there are other possible logical conclusions depending on your philosophical foundations, e.g. the brain could be a receiver for consciousness which is translated into our worldly experience from somewhere else.

[vouwfietsman]

"an", not "the" alternative.

Consciousness can be not-emergent but also not metaphysical, think sci-fi-type undiscovered physics or matter.

[kelseyfrog]

Another alternative is that consciousness exists on the map, and unfortunately we're confusing that with the territory.

[fc417fc802]

Aren't you saying the same thing? It seems like the metaphysical and the map would be analogous here.

Of course both of those suffer from the recursive problem of just kicking the can one level up. But I guess that's fundamentally unsolvable so who cares.

[kelseyfrog]

I feel like someone confusing the map and territory can exist without needing to invoke the metaphysical. Maybe, I'm misunderstanding tho

[eternauta3k]

I don't know if most people would dismiss the sentence "all matter has a sort of proto-subjectivity very different from ours but which gives rise to ours". And it solves some problems (introducing others).

[doctoboggan]

Panpsychism is certainly an interesting idea but I wouldn't consider it a popularly held view.

[trick-or-treat]

That's because Panpsychism is silly.

[IsTom]

I don't think it's any more silly than the alternatives.

[eszed]

I'd expand a little bit to say inevitably emergent property. That is to say, if you create a sufficiently complex information-processing network, some level of consciousness will result. With regards to current AI, we're a fair way away from building something with enough connections, but we'll get there.

One thing that gives me pause about the inevitability hypothesis is that the type of connection, or manner of information processing, may matter: there might be something about neurons that isn't (currently) reproducible in silicon. I don't know, and there's not (yet) any evidence for or against, but it at least seems like plausible speculation. We just don't know enough about any of this right now.

[slopinthebag]

> That is to say, if you create a sufficiently complex information-processing network, some level of consciousness will result.

Why is that?

[eszed]

Aye, that's the question, and we don't know: everything in this sub-thread my comment kicked off is speculation. The thing is that several large groups of (or at least led by) fundamentally reckless people are racing to build machines that will test these hypotheses, with little regard for the consequences. The rest of us are scrabbling around in their wake wondering what's going to happen next.

[slopinthebag]

Well, I think the "brain as an antenna" theory is also plausible given your preconditions.

But I think my issue with the emergence theory is that it seems to imply to me that consciousness is non-physical and non-local. So what entity is actually experiencing the consciousness? It's not that I believe consciousness is physical and local, but people who make the emergence argument seem to believe it is and I can't figure out how that is supposed to work.

[doctoboggan]

Why would emergence imply anything about non-physicality and non-locality? Temperature is a another common example of an emergent phenomena. An individual atom doesn't really have a temperature, only a large group of them do. But you wouldn't say temperature is non-physical and non-local, would you?

[davrosthedalek]

I am not sure, but I think you might misunderstand what emergent means. Take chaotic systems, in the math sense. Chaos is a well defined property of, say, iterated dynamic systems.

A linear dynamic, say x_n+1=lambda x_n, or x_n+1=(1-x_n), is never chaotic. But if you multiply them, x_n+1=lambda x_n (1-x_n), it depends on lambda if the system is chaotic.

None of the components are chaotic. But for specific combinations, chaos emerges as an property.

In physics, the mass of mesons and the nucleon is emergent. It's completely different from the constituents' mass. Different from an atom, where its mass is very close to the mass of its nucleus and its electrons.

[eszed]

Not a gotcha at all, but I don't have a satisfying answer, nor am I confident there even is one. Best I can do is to say that I think consciousness and sense of self are at the very least closely related, and perhaps the very same phenomenon. "I" am the entity that realizes my own consciousness; consciousness is the qualia that makes "me" separable from all other entities.

Or something like that. This gets to the "dorm room bullshitting" level right quick.

[slopinthebag]

Yeah, I guess what I'm trying to ask is that if it is an emergent property but not a physical part of the brain, doesn't that imply something metaphysical about consciousness? Almost as if it's a non-physical phenomenon? At least when I hear people talk about emergent behaviour I see it as a refutation of the spiritual, but to me it seems like it actually implies we have a "soul".

Idk, it's really hard to articulate my thoughts here and yes it is pretty close to the conversations I had in college on various substances. Lol.

[eszed]

I don't know if it implies that. Someone up-thread mentioned temperature as an emergent property: individual atoms don't have it, but a sufficiently large group of them do. That would, I guess, make temperature meta-physical in the most literal possible sense. That's not how we typically think of that term as applied to consciousness or soul or whatever, but I'd agree it fits, without implying any kind of specialness beyond that.

[pixl97]

Is a video game a physical property of a computer?

[bulbar]

We have general purpose hardware and we have hardware that's hard wired for specific purposes like ASICs and we have everything in between.

And we are only doing it for a few decades. Evolution had million of years of "try and error".

[slopinthebag]

Yes

[pixl97]

So then we can assume consciousness is a physical property of brains, as we can measure the electrical signals of consciousness versus when we are unconscious.

[therealdrag0]

Those are the questions and there’s stacks and stacks of philosophy pages written about it. Go have a whirl.