[indoordin0saur]

I often hear the assumption that LLMs can or will become conscious because consciousness is likely substrate independent. The idea being that our brain is just a computer made out of meat and it doesn't do anything that can't be precisely simulated by a silicon computer.

But I wonder if some of the magic in the human brain is its analog nature. Chemical signaling and impulses of neurons interact with each other with waveforms that have theoretically infinite detail. In contrast, computers discrete, quantized values, storing no more detail of a signal than what is needed for the programmers desired task. Is there perhaps something about the continuous and chaotic nature of analog data that could give rise to consciousness? If so, it seems like it would preclude consciousness ever being seen in digital form.

[cwmma]

Probably not, we're nowhere near the complexity of the human brain yet, there are also quantization limits to the human brain (i.e. molecules, quantum physics, etc) so to characterize them as having infinite detail is probably a bad modal.

If I'm going to be honest most of the people who advocate this type of thing tend to be, shall we say, crypto-duelists who really believe in a soul but not like intellectually but intuitively and keep trying to come up with excuse with it's not just meat. So like you can find philosophers advocating stuff like this but they tend to have a bit of an agenda.

[AndrewKemendo]

You nailed it which is that almost everyone I’ve met seems to believe in some version of dualism

If that’s even subtly your position then there’s no way to have a productive conversation about perception, reality/truth, epistemology and especially consciousness

It’s honestly maddening cause I’ve had great conversations about this with educated people, and all but only a handful, collapse into the other person relying on some nonfalsifiable dualist argument

[indoordin0saur]

> collapse into the other person relying on some non-falsifiable dualist argument

Are there any non-duelist, scientific theories out there that could plausibly be tested? I can't say I've seen any but if you know of any then I'm curious to hear about them. From what I've seen, anyone trying to explain phenomenal consciousness in scientific objective terms falls into at least one of these three strategies:

1.) Saying that consciousness "arises" inevitably or is an emergent phenomenon of a complex information processing system. There are a number of theories along these lines but they aren't falsifiable from what I've seen and usually at some point rely on some magic unexplainable step or are actually dualist.

2.) Defining consciousness just as the easily explainable stuff via biology, such as being awake vs. asleep.

3.) Dismissing the idea that subjective experience exists at all. I sometimes wonder if people arguing strongly for this are something like a philosophical zombie and there's nothing inside them experiencing.

[FloorEgg]

My intuition is that a single unit of consciousness is a wave/particle interaction, and that the kind of consciousness we experience has a particular volume and shape, some parts of which are fairly consistent between people and some parts which vary considerably. The more volume of consciousness the more diversity can fit into it.

My mental model includes integrated information theory and Karl friston free energy principle, and something about temporal computation on a physical graph structure.

Which camp would this fall into? 2 seems closest but kind of undersells it...

[indoordin0saur]

Even with information integration theory it doesn't explain any of the why or what is happening of subjective experience and there still seems to be a leap in there that has a mystical or mysterious aspect to it. Basically [specific mechanistic process explainable with physics] -> [mysterious missing step] -> [consciousness]

[AndrewKemendo]

I’m both confused and intrigued at this question

Every scientific theory that is predictably/measurably correct is not dualist by nature.

Dualism assumes that there’s a non-measurable variable (usually undefined) but nevertheless has a causal input in action determination

Dualism is precisely non-scientific if you are using the classic Baconian-Khaneman you epistemological process because it introduces variables that cannot be measured

[indoordin0saur]

Well do you know of any non-duelist and theoretically testable theories on consciousness? I'm just pointing out when the non-duelists are completely stumped no one should be surprised if alternative ways of thinking get thrown out there.

[AndrewKemendo]

There’s not consensus on what consciousness is defined as.

There are a lot of people out there who have their own versions of definitions but again there’s no consensus, I personally do not throw my hat in with any written definition other than my own and I’m hesitant to share that.

Saying that “dualists have an answer” doesn’t actually solve it if there’s no shared definition

Much like the term “intelligence” has no consensus, so being able to determine what is “artificial” versus “not artificial” intelligence continues to be this philosophical or almost religious position.

Creating a consensus on what the term “consciousness” means mechanically, would effectively destabilize the entirety of society. Imagine if the consensus definition of consciousness is applicable to all mammals. That means that there would be valid justification then to make it illegal to harm any mammal. You can see why this would incentivize people to not come to consensus on these things.

So in my perspective it is not socially feasible to find consensus and therefore a way to test it because religious leader might have incompatible definitions of consciousness than let’s say an epilepsy doctor.

As much as a scientists want to actually make progress in the world ultimately what holds us back are somewhat arbitrary social conventions because reasons.

[FloorEgg]

I'm not sure if it's what you're looking for, but Karl friston free energy principle seems to me the most scientific theory of consciousness. (At least that I've come across)

[km144]

I'm basically restating what you said, but it's amazing to me that the vast majority of people you will meet, even educated people, are casual dualists and free-will libertarians. If they happen to acknowledge materialism in some way (i.e. the acceptance of the idea that the brain's processes are just the interaction of physical matter), there is still zero chance they draw determinist conclusions from that acknowledgement. But I guess that tracks, given that most professional philosophers are apparently compatibilists for some reason I have never understood (the arguments get really confusing).

[AndrewKemendo]

I’ve studied this in extreme depth and my conclusion is that at a certain point most people just give up and take a position that gives them some kind of mental relief

For people who can’t self-delude, like Godel and Schopenhauer and Tesla, they kind of just go “mad” because it’s a giant epistemological hole they can’t solve.

Even the smartest the scientist are always going to choose self preservation in their cognitive capacity so they don’t feel bad over feeling bad and living in the contradiction

This is why it’s important for scientists to study Camus IMO

[reasonableklout]

How are you measuring the complexity of the human brain?

[spandrew]

I wish people would knock this off. There is zero path to AGI at the moment — and all the Anthropic/pentagon Sam Altman/AI-skynet stuff is scaring people into being fearful (and outright ignorant) to the actual uses of this probabilistic NPL tool.

You can't use AI atm without a human with proper knowledge spot checking and directing it. We should market it that way.

[lumost]

The current theme is that agi may not be definable, and an ai which matches humans on all economically relevant tasks is close enough for business purposes.

Billions spent on RL may be good enough to beat human performance.

[loudmax]

I'm very open to the idea that consciousness is substrate independent. I have a hard time seeing why molecules could produce consciousness from an electro-chemical path, but not from a purely electrical path. Having said that, it should be very clear that LLMs are not conscious.

LLMs process language. I'd even go so far as to say that LLMs "think" and "understand", or at least, they produce a facsimile of thinking and understanding such that it's useful for us to reason about LLMs as if they think and understand. We're not used to interacting with a non-human entity with the capability to process language, so it's easy to ascribe human traits to these things. But their "minds" (insofar as they have anything like a mind) are completely different from ours. These things have language without consciousness.

Chimpanzees are conscious. Dogs are conscious. Maybe ravens and cephalopods? Who knows. These animals do have minds much like ours. Higher order animals are conscious even if they don't have language.

[Gooblebrai]

> I have a hard time seeing why molecules could produce consciousness from an electro-chemical path, but not from a purely electrical path.

Do you have any particular reasoning behind this? I could equally say that I have a hard time seeing why molecules could produce consciousness from a purely electrical path.

[echoangle]

If we want to be really pedantic, a computer does not necessarily need to be digital. You could build a consciousness-coprocessor with analog circuitry that brings in what you say could be needed.

But honestly I doubt that there is a real requirement for that, surely you could just increase the resolution you're running the simulation at until the difference decreases sufficiently. Imagine using 256 bit floats for example.

[nomel]

IBM has made a few analog chips like this [1]. If I remember correctly, one was even pulse time based, like neurons.

https://research.ibm.com/blog/analog-ai-chip-inference

[fcanesin]

There are quantized processes in the brain and there is also analogic computing. So either way is just a matter of time science gets there.

[willmadden]

It’s not yet clear whether consciousness can be fully explained in physical terms. There is strong evidence linking it to brain activity, but we don’t have a complete theory of how subjective experience arises. It’s possible our current frameworks are incomplete and there are other forces we cannot currently measure in play.

[varjag]

Each and every one of many billion transistors in a GPU works due to certain quantum effects on P-N junction. That does not mean GPU computation is nondeterministic, unknowable or magic.

[etbebl]

Sure, but it's designed to minimize the chance that a quantum fluctuation could change the outcome of a computation, right? Whereas in the brain that might not be the case. A lot of the "interesting" neural activity (e.g. relating to decision making, language, etc) happen in highly sensitive dynamical regimes: on a threshold of firing, or activating one neural population vs another. (Arguably you can get the same effect in an artificial network by adding true random noise though!)

[varjag]

It's not implausible that evolutionary pressures made the brain robust enough to withstand parasitic signals. We know that most people think in similar, predictable ways.

[lstodd]

I'm tired of those references to "most" people.

Maybe let's rename AI to AMI: Artificial Mostly Intelligence ?

[varjag]

Are we in "define is" territory now?

I used a qualifier because (as you certainly know) people will not pass a chance to nitpick. I have no energy to argue about neurodivergent or Ramanujan, even though I still believe the basic physics of their thinking is identical. Hence "most".

My argument was straightforward to follow but if there's something unclear let me know and I'll chew it down.

[Kattywumpus]

One analogy might be to the misleadingly named motion picture. We all have heard the stories of early cinema-goers who dove out of the way when the train on the screen barreled towards them. It's easy to make the mistake of thinking of these images on the screen as the real thing -- even the mistake of thinking that they move.

But they don't move. They can't. In the end, a film is just a series of still images. The images are projected in such rapid succession that they fool us into seeing things in motion that are not actually in motion. And if we slowed the film down enough, from 20 frames a second, to say, a frame every 20 seconds, well... the illusion would stop. We'd see each discrete still image for exactly what it was.

This is the same way computers work. Every tick of the clock, we get a new static image in memory. But those images are all static, as still as the images on film. And no static matrix of information can be conscious, just like no still image can move.

This is radically different than the neurons in our brains, which are each living things, sensing things, with a continuous analog existence. Whatever we feel comes from their lives. We can measure the things they do with their lives, but we shouldn't mistake the measurement for the thing being measured. A ruler is a foot long, but it’s not ticklish.

[Aperocky]

Consciousness is substrate independent, but that's not the differentiator here:

My toddler son is conscious, he feels happiness and sadness and have urges and impulses without needing to know the entire history worth of writing from human civilization.

A dictionary is not, and a LLM sized dictionary with optimized query is still not conscious.

It doesn't have anything to do whether consciousness is substrate independent, or even analog vs discrete, the overall implementation just aren't parallel.

[otterdude]

Wouldn't a conscious entity be able to grow its intelligence over time independently?

LLM's require offline training and dont actually learn from their "lived / sessions / chats ect". Those can be used for training data but its not like its an implicit part of the technology.

For this reason I would say LLM's are not conscious, or automatically conscious.

[andrewla]

I think philosophically this comes down to the Universal Church-Turing Thesis.

If you believe that there is computation being done by nature/physics that is strictly more powerful than what a Turing machine is capable of, then consciousness might be beyond the reach of a silicon computer.

I personally believe that the UCTT is very likely to be true, not least because we have not yet proposed an oracle that is physically feasible and yet is beyond the capabilities of a Turing Machine.

That said, in the space of feasibility there are open questions -- even if P != NP, a non-deterministic Turing machine can be simulated by a deterministic one, just potentially with an exponential increase in computation time or space.

[boppo1]

How certain are we about the theory our minds waveforms are continuous? Can we prove physical continuity, or just up to planck-length resolution?

I'm poorly educated on this, these are sincere questions. They are not intended as rhetorical regarding the point of the prior post.

[achow]

It is perhaps very relevant what Chris Olah (Anthropic co-founder) said to pope..

I am a scientist. I lead a research team that studies the internal structure of these models—what is actually happening inside them. And I will be honest: we keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease.

https://www.anthropic.com/news/chris-olah-pope-leo-encyclica...

[boppo1]

Yeah that reeks of marketing to me. This guy may believe all those things, but his getting paid may also depend upon his believing them.

[ch4s3]

This is an incredibly strong claim and I’ve never seen any particularly strong evidence for them beyond “trust me, I’ve seen into the abyss.”

[mrdomino-]

I think what you've said here is unfair and overly cynical; nowhere have I read Chris or any of the Anthropic people make the claim that LLMs definitively are conscious. What they say is that there is increasing uncertainty, and evidence - for which they show receipts - backing up that uncertainty.

Personally, I have been unconvinced by any of these definitive yes or no answers to "are LLMs conscious?" - in both directions. The "they are not / can not be conscious" side relies too heavily on mechanistic reductionist arguments that can apply equally to neurochemical processes in the human brain - and yet, it seems that humans, with brains made of these neurochemical processes, are conscious. At the same time, the "they are definitely conscious" answers seem to generally rely too heavily on self-deception and lack of reality testing.

To be able to say something definitive here, it seems to me that one would need to say definitively what this experience is. I have not heard any of the loud voices in the arena - not Chiang, not Giulio Tonini, not Karl Friston, etc - do so. Therefore I find Anthropic's uncertainty, and careful, caring investigative process, well grounded given the evidence.

[ch4s3]

My aim isn't cynicism so much as an interest in clear an evidence based information about these systems which are clearly important technological developments for the future. It may turn out that there is something analogous to cognition going on and that would be quite significant and require us to think differently about the technology, but we need to be very sure of something like that.

> Therefore I find Anthropic's uncertainty, and careful, caring investigative process, well grounded given the evidence.

I'm inclined to agree mostly, but from the outside its impossible to separate marketing, belief, scientific inquiry, and just plain enthusiasm. Its all to opaque.

[mrdomino-]

I will concur that it's too opaque.

[irishcoffee]

The point here, Chris Olah is literally claiming he has seen into the abyss. Who are you disagreeing with?

[ch4s3]

I'm saying that the burden of proof is on Chris Olah.

[astockwell]

This is the fundamental trust challenge with private/profitability-driven 'science'.

[mrdomino-]

And it is worth noting, I think, that Olah's message to the pope is centrally making that point: that Olah and his peers are within incentive structures that distort their actions and understanding, and therefore that dialog with those outside of such incentive structures is necessary (not sufficient) for their outputs to be trustworthy.

[ikrenji]

well of course anything from anthropic on the topic is going to be bombastic and hyped to the moon

[ezst]

I mean, this reminds me of the early imagenet days, when people were first trying to explain the unreasonable efficiency of neural networks at interpreting content in images. They found out that, after enough backprop, some layers of the network became specialized in contours extraction, shape extraction, etc, in ways that can be seen as analogous to earlier CV techniques (Canny/Hough transforms, ...).

Later, Google was having fun feeding whole bunch of youtube content to artificial neural networks, unsupervised, and figured that certain parts of the network would, too, specialize, only to have the activation functions be run backwards and render an abstract image of a cat¹.

None of that is terribly new or surprising for anyone having studied and dealt with neural networks. The only difference today is that the field has completely flip flopped from approaching the subject with scientific rigor and cautious excitement to being a clueless billionaire infinite money printing machine fed on deceiving anthropomorphism and FUD.

¹: https://blog.google/innovation-and-ai/products/using-large-s...

[insanitybit]

Total nonsense.

[tooheavy]

I think the representation in a computer, the fact that it is merely stored instructions and data, destroys everything but domains of simulation and emulation.

[nomel]

"I think the representation in a brain, the fact that it is merely stored activation potentials in neurons firing based on ionic accumulations at their synapses, destroys everything but domains of simulation and emulation."

;)