[nathan_compton]

I believe it is eminently credible draw a strong association between consciousness and the physical activity of the brain since it is relatively well backed up by scientific observation that there is a one to one correspondence between conscious experience and brain activity. Although we still don't understand precisely how the physical activity creates qualia, I think its perfectly reasonable to say that studying and understanding brain activity constitutes studying and understanding consciousness.

[ToValueFunfetti]

We don't understand consciousness as it pertains to the underlying question; knowing that brain activity can produce consciousness does not get us any closer to knowing that a matrix multiplication can't.

[nathan_compton]

True, but I don't dispute that in principle a lot of matrix multiplication could produce consciousness. I just suggest that an honest appraisal of brains and LLMs suggests little to no consciousness on the part of the latter.

[Last5Digits]

If we use your metric, then an honest appraisal of brains and computers suggests little to no mathematical ability on the part of the latter either. If we assume that a similar medium or structure is necessary for similar results, then it should be highly improbable that a bunch of semiconductors could ever perform even simple math, since they are very structurally dissimilar to the human brain.

[nathan_compton]

Only if you insist on thinking of brains and ICs as magical mysterious objects about which nothing can be said. We understand how both of these objects work to one degree or another. My point is that it is precisely the understanding of both phenomena which suggests that LLMs are not conscious or, arguably, intelligent.

[Last5Digits]

The "to one degree or another" is doing all the work in this argument. Does my knowledge of how a full adder works now grant me the ability to discern malware at a glance? Similarly, should we start dismissing psychologists because your average neurologist can just cure depression and other mental issues? Should we do the same with sociologists or economists? Maybe even neurologists could be replaced by physicists or mathematicians.

Or maybe the abstract, high level understanding the brain provided by psychology is enough to explain its dynamic behavior? Maybe I can become an expert in IC design by learning React?

We know how neural nets work on a fundamental level and we know how they work on multiple levels of higher abstraction, yet explainability is one the biggest problems in machine learning right now. These models can solve complex problems which computer scientists long struggled to develop algorithms for, even though every aspect of them, except emergent behaviors due to complex interactions, is known by us.

The issue is that consciousness is a strongly emergent property - touching every level of abstraction and comprising patterns from the specific to the general. Knowledge of how a system works on the ground level or how it works on some coarse levels of abstraction, does not allow you to classify it as conscious or unconscious.

Additionally, consciousness is ill-defined. There is no agreed upon definition that is free of contradictions, does not accidentally include systems that we would not see as conscious or does not accidentally exclude a significant portion of humanity.

I invite you to think up some properties of the human brain that you would classify as essential for consciousness to emerge, and then try to think up exceptions. I'm very confident that you can come up with at least one for every single property.

[nathan_compton]

Yes, yes, yes, but pretending that we know nothing at all about consciousness or that nothing can be said about the likelihood that an LLM has it or other properties is absurd in the extreme. There are genuine limits to stupidity.

[JohnFen]

Sure, but none of the means we know what "consciousness" is.

[nathan_compton]

Every account of the universe is grounded in brute facts, for which there is no justification. That hardly means we can't claim to understand things. I would say we understand consciousness more or less in the same way we understand nuclear physics: we have a very compelling ontologically flavored justificatory framework which both allows prediction and makes theoretical sense. We know quantum field theory is not the right theory of the universe. We may never know the fundamental theory. But it would be ridiculous to say "we know nothing about how the nucleus works."

[JohnFen]

> I would say we understand consciousness more or less in the same way we understand nuclear physics

But we know what nuclear physics is. We don't know what consciousness is. I'm not asking how consciousness works, I'm asking what consciousness is.

[nathan_compton]

If you study enough theoretical physics, you'll begin to wonder what nuclear physics is, I can assure you.

[halostatue]

I think that we know what it isn’t, even if we can’t fully define what it is.

[c22]

I know what consciousness "is." I'm experiencing it right now.

[dekhn]

Ah. I use the term "the subjective experience of consciousness". IE, what you are experiencing could really just be a VR drug hallucination, completely unrelated to anything else, or an epiphenomenon of a completely mechanistic universe.

[JohnFen]

I would argue that if you can't communicate it to another, then you don't know what it is. You only know what it's like to experience it.

[c22]

I guess it depends on what your definition of is is!

To put it another way, either you also have consciousness, in which case I have explained it, or else I can't explain it to you because you don't.

[Bridgoo]

Prove it

[c22]

No, it's entirely self-evident. You prove I don't.