[SamBam]

> I agree that there is a significant sliver of a philosophical problem which remains stubborn (how precisely does physical activity produce qualia)

But that IS the definition of consciousness! This is like saying "We understand practically everything about airplanes, except how they stay in the air."

Nothing discussed in neuroscience is relevant to understanding what consciousness IS (which is the question posed above). Finding out that stimulating such and such a region makes us sad, or that this bundle of nerves activates before we're consciously aware of a decision doesn't tell us anything about consciousness itself. We've known for hundreds of years that there is a relationship between the brain and consciousness, finding out more details doesn't answer the question.

(Now, whether consciousness is necessary for AGI is a separate question.)

[munificent]

> This is like saying "We understand practically everything about airplanes, except how they stay in the air."

Even that is a sufficient level of understanding to correctly determine that a motorcycle is not an airplane.

While we might not have a complete picture of what consciousness entails, we can at least list some necessary conditions for it to arise. Any system that lacks those conditions can at least be proven to not be conscious.

With LLMs specifically, I think there is a very strong argument that they are not and cannot be conscious at all, regardless of how big of a corpus you throw at it or how many parameters it has. Emily Bender explains it well here:

https://medium.com/@emilymenonbender/thought-experiment-in-t...

[kbenson]

> Even that is a sufficient level of understanding to correctly determine that a motorcycle is not an airplane.

But are we talking about airplanes, or are we talking about "flying"? Airplanes fly, motorcycles don't. Do hot air balloons fly? Or is floating not flying?

Neuroscience tells us about human and similar consciousness. Maybe Ll biological consciousness, but maybe not even that. Are we sure we're not exploring a subset of consciousness though, and other variations exist that were unaware of and will catch us off guard because we haven't encountered them (or recognized when we have)?

I think that's the important question here, and it goes beyond LLMs, because whether they can or can not achieve consciousness doesn't mean something else will follow the same path.

[munificent]

> Are we sure we're not exploring a subset of consciousness though, and other variations exist that were unaware of and will catch us off guard because we haven't encountered them (or recognized when we have)?

I don't know of any general principle one could use to determine if system X has or doesn't have property Y if you don't at least have some definition of Y.

Do microwaves flern? Can fish frabulate?

[nathan_compton]

I believe it is eminently credible draw a strong association between consciousness and the physical activity of the brain since it is relatively well backed up by scientific observation that there is a one to one correspondence between conscious experience and brain activity. Although we still don't understand precisely how the physical activity creates qualia, I think its perfectly reasonable to say that studying and understanding brain activity constitutes studying and understanding consciousness.

[ToValueFunfetti]

We don't understand consciousness as it pertains to the underlying question; knowing that brain activity can produce consciousness does not get us any closer to knowing that a matrix multiplication can't.

[nathan_compton]

True, but I don't dispute that in principle a lot of matrix multiplication could produce consciousness. I just suggest that an honest appraisal of brains and LLMs suggests little to no consciousness on the part of the latter.

[Last5Digits]

If we use your metric, then an honest appraisal of brains and computers suggests little to no mathematical ability on the part of the latter either. If we assume that a similar medium or structure is necessary for similar results, then it should be highly improbable that a bunch of semiconductors could ever perform even simple math, since they are very structurally dissimilar to the human brain.

[nathan_compton]

Only if you insist on thinking of brains and ICs as magical mysterious objects about which nothing can be said. We understand how both of these objects work to one degree or another. My point is that it is precisely the understanding of both phenomena which suggests that LLMs are not conscious or, arguably, intelligent.

[Last5Digits]

The "to one degree or another" is doing all the work in this argument. Does my knowledge of how a full adder works now grant me the ability to discern malware at a glance? Similarly, should we start dismissing psychologists because your average neurologist can just cure depression and other mental issues? Should we do the same with sociologists or economists? Maybe even neurologists could be replaced by physicists or mathematicians.

Or maybe the abstract, high level understanding the brain provided by psychology is enough to explain its dynamic behavior? Maybe I can become an expert in IC design by learning React?

We know how neural nets work on a fundamental level and we know how they work on multiple levels of higher abstraction, yet explainability is one the biggest problems in machine learning right now. These models can solve complex problems which computer scientists long struggled to develop algorithms for, even though every aspect of them, except emergent behaviors due to complex interactions, is known by us.

The issue is that consciousness is a strongly emergent property - touching every level of abstraction and comprising patterns from the specific to the general. Knowledge of how a system works on the ground level or how it works on some coarse levels of abstraction, does not allow you to classify it as conscious or unconscious.

Additionally, consciousness is ill-defined. There is no agreed upon definition that is free of contradictions, does not accidentally include systems that we would not see as conscious or does not accidentally exclude a significant portion of humanity.

I invite you to think up some properties of the human brain that you would classify as essential for consciousness to emerge, and then try to think up exceptions. I'm very confident that you can come up with at least one for every single property.

[JohnFen]

Sure, but none of the means we know what "consciousness" is.

[nathan_compton]

Every account of the universe is grounded in brute facts, for which there is no justification. That hardly means we can't claim to understand things. I would say we understand consciousness more or less in the same way we understand nuclear physics: we have a very compelling ontologically flavored justificatory framework which both allows prediction and makes theoretical sense. We know quantum field theory is not the right theory of the universe. We may never know the fundamental theory. But it would be ridiculous to say "we know nothing about how the nucleus works."

[JohnFen]

> I would say we understand consciousness more or less in the same way we understand nuclear physics

But we know what nuclear physics is. We don't know what consciousness is. I'm not asking how consciousness works, I'm asking what consciousness is.

[nathan_compton]

If you study enough theoretical physics, you'll begin to wonder what nuclear physics is, I can assure you.

[halostatue]

I think that we know what it isn’t, even if we can’t fully define what it is.

[c22]

I know what consciousness "is." I'm experiencing it right now.

[dekhn]

Ah. I use the term "the subjective experience of consciousness". IE, what you are experiencing could really just be a VR drug hallucination, completely unrelated to anything else, or an epiphenomenon of a completely mechanistic universe.

[JohnFen]

I would argue that if you can't communicate it to another, then you don't know what it is. You only know what it's like to experience it.

[c22]

I guess it depends on what your definition of is is!

To put it another way, either you also have consciousness, in which case I have explained it, or else I can't explain it to you because you don't.

[Bridgoo]

Prove it

[c22]

No, it's entirely self-evident. You prove I don't.