[siglesias]

I think one of our greatest risks is precisely the opposite--anthropomorphizing computer programs because they produce certain behaviors. They will very clearly exploit us by claiming they're conscious, scared, and in pain. Claiming something is conscious because it behaves realistically isn't a scientific position; it's bad metaphysics and it's superstition. Brains cause minds and brains cause consciousness. Consciousness is a biological, physical phenomenon like any other: photosynthesis, digestion, bioluminescence. You don't get the physical phenomenon of consciousness by finding a program that behaves realistically and implementing it in hardware (which by the way, theoretically can use any physical mechanism, not only silicon). At best you're creating a simulation. If you simulate a rainstorm, there is no physical wetness. In the same way, to suppose that computers are literally conscious because they have a facility with language is a deep fallacy and an illusion. Artificial brains that duplicate the brain's causal powers of consciousness are theoretically possible, but programs are not causally sufficient to make computers into conscious artifacts.

[epgui]

> it's bad metaphysics and it's superstition

And yet it's the same thing I do when I talk to you, to my family and friends, or to strangers: I assume that other people's experiences are just as valid and real as mine, even though it's not something I can ever verify or validate completely.

> Consciousness is a biological, physical phenomenon like any other: photosynthesis, digestion, bioluminescence

Consciousness is unlike the other things in the list in that it's not "a thing", but rather "a collection of things" that all work together to produce emergent behaviour. You could say that it's similar to digestion in that sense, but for some reason we don't usually apply the same kind of mysticism to digestion.

[siglesias]

Your friends (and animals) have brains, which produce consciousness, which you take for granted when you interact with them. That’s not a superstition. Can you say more about “a collection of things”? What causes that collection of things?

[epgui]

Brains are more like a collection of interconnected microservices than a big monolith. What "causes" that? Well, depending on what you're really asking, it's a self-organizing organ, so you can point to an embryological explanation or to an evolutionary explanation or both... But none of that changes the fact that an adult fully-formed brain processes information, and that all of our experiences *emerge* from this information processing. Everything. Your emotions, your thoughts, your impulses, your biases, all of it.

If you're coming at the subject with an engineering background, a good place to gain a broader perspective would be a functional neuroanatomy textbook (eg.: Blumenfeld).

[siglesias]

I just want to flag that assuming that humans and other animals are conscious because they have brains is not superstition in the same way that inferring consciousness from behavior is. I assume you concede these are two different leaps of inference.

It’s not clear how it follows from the brain being interpreted as an information processor that the phenomenon of consciousness emerges. Many objects can be interpreted as processing information (even computers not running AI); are you asserting they’re all conscious as well? How does nature decide what counts as information? The only causal power I see in nature is physics, not information, so this account isn’t super coherent to me.

[epgui]

> are you asserting they’re all conscious as well?

I think the only part you're not understanding from what I said is the keyword "emergent". If you understand what that word means, you will understand the difference between a calculator and a sufficiently large neural net, or a neural net that is organized in a particular way.

- https://en.wikipedia.org/wiki/Emergence

> How does nature decide what counts as information?

Nature doesn't "decide" anything, it just "is". "Information" is a human construct, but it can be measured objectively (see Shannon Entropy for the tip of the iceberg).

> The only causal power I see in nature is physics, not information

Then you're missing some perspective.

- https://physics.stackexchange.com/questions/2685/what-is-inf...

Not to over-anchor on just a few people or concepts, but see for example (I am sharing pop science publications here but there is no shortage of more serious materials on the subject):

- https://mindmatters.ai/2021/05/it-from-bit-what-did-john-arc...

- https://bigthink.com/13-8/information-central-physics-univer...

You may even have heard of the black hole information paradox or quantum "states" (which is often studied from an information theory perspective):

- https://en.wikipedia.org/wiki/Black_hole_information_paradox

- https://en.wikipedia.org/wiki/Quantum_information

Even if information was purely an imaginary concept (I withhold judgment on this), it's probably the most fundamental language we have to describe physics as a whole.

[siglesias]

I appreciate the helpful links, and I do have training in physics so the sense in which information is described in physics is not lost on me (it's essential to even 19th century thermal physics). However, there is a discrepancy between information as it relates to quantum states, entropy, and information as it relates to interpreting bits in a computer program. Computers purely operate on bits in a computer program, relying on the underlying physics of the implementing substrate to carry out the operations. The actual physical process carrying out the steps of a program can rely on anything with causal powers to run a Turing machine--it can be an elaborate system of beer cans, mice and cheese, water pipes, or silicon. On the level of physics (quantum, entropy, whatever), the actual amount of information could be constant along a range of programs, some of which can pass the Turing test for humans, but most of which won't. This would suggest to me that if consciousness names a physical phenomenon, it's not to be found in a description of a computer program--therefore programs are not causally sufficient to produce consciousness, only a particular pattern of behavior salient to humans.