[monstertank]

I never understood this. Evolution basically says humans are conscious matter...why would that not be replicatable via an intelligent designer (humans)?

Randomness created consciousness from a single cell...why would that be the most efficient solution?

[jugg1es]

We can't replicate something we don't understand. The mechanism for consciousness is not understood at all right now and could actually be based on quantum effects that we haven't detected yet. It's also possible that it is only achievable in an organic machine. Until we understand how consciousness actually arises, the best we can do is a simulacrum of it.

[EasyMark]

If it walks like a duck, and quacks like a duck, it's a duck. I don't understand this "we have to understand consciousness first" stuff gets so much discussion. If it acts human, and is indiscernible from human or human like, it'll fit our definition. I don't understand why "organic machine" matters, it's the "machine" part that matters, and we are machines. I think all that needs to be seriously considered. Is it ethical to end such an intelligence because it could easily evolve far quicker than us and displace us? Do we just let "natural selection" happen? big questions, but I don't think we'll understand and have universal understanding of consciousness before we stumble onto the first AGI

[silver_silver]

Something which pretends to be something else, no matter how convincing, is still essentially different. A deepfaked video could be bit for bit identical to what the genuine recording would have been and it still wouldn’t reflect reality if the content never happened.

[Thiez]

But since we can't yet define consciousness, how do you know the AGI is just 'faking it' instead of being the real deal? What are human brains but a bunch of neurons pretending to be a person?

[silver_silver]

That’s fair, we can’t even prove humans aren’t like you say - although the brain is vastly more complex than a neural network analogy lets on. I suppose once these models become sophisticated enough we’ll have to treat them as conscious until it can be proven one way or another.

[kaibee]

> That’s fair, we can’t even prove humans aren’t like you say - although the brain is vastly more complex than a neural network analogy lets on. I suppose once these models become sophisticated enough we’ll have to treat them as conscious until it can be proven one way or another.

I suspect that a lot of the complexity is legacy code and/or incidental to the implementation. Cells have to do a lot of things just to continue living. Likewise, a lot of what the brain does is just control the very complex biochemical reactor that it lives in. ie: whales have big brains relative to humans (though they are a much smaller % of their body).

[jugg1es]

I guess I just disagree with you. My background is neurobiology and biological systems are just so fundamentally different than conventional computing architecture in so many ways. I just don't believe that a true AGI is achievable as software alone. We might get something that "looks like a duck" but until it actually is a duck, I don't think it will be AGI.

[nox101]

I replicate things I don't understand all the time. I replicate bread. I haven't a clue why it turns into bread. I just follow the instructions and it becomes bread.

We don't need to know how to simulate consciousness. We only need to simulate atoms. The rest will fall out. Maybe there's a shortcut. Maybe we only need to simulate molecules. Maybe only cells. But one way or another, we can skip the understanding consciousness part.

[jugg1es]

I think there might be more to consciousness than just simulating the physical components of the brain that we currently understand. There are theories that neurons or clusters of neurons are leveraging quantum superposition in order to function. If that turns out to be true, then a simulation of the physical parts won't be enough for AGI. It would also mean that we actually do need to gain a better understanding of how the brain works in order to achieve AGI. I am skeptical that we can achieve AGI with software alone.

[gls2ro]

If you think about the bread example, what you do is to follow a very well known recipe even if you don't understand yeast.

In case of consciousness, we lack both types of knowledge: - What consciousness is - To use your example we don't know what bread is - How to create conciousness - To use your example we don't know a recipe for bread

To make this a bit more like a story, imagine we all got to a new place and tasted a thing called bread. We are not even sure we all tasted the same thing, nor can we agree on how to describe it, except we call it the same "bread." we don't have any recipe for it or know its ingredients. To make this even stranger we are not even sure bread is something that we are supposed to eat or it is something that we wear. Maybe it was some kind of cloth that as a strange experiment it was given for us to taste.

And now we are trying to create it. It could be that taking enough time and energy and people that are trying to create this thing called bread, we might, in the end, arrive at something that, when tasted by everyone, we can agree it is bread, but that seems less probable until we agree on a description of it.

[nox101]

I would make the analogy to gunpower. We knew how to make gunpower from raw ingredients for nearly 3000 years before we knew how and why those ingredients actually caused the reaction they do.

I expect it may be the same for consciousness. We'll put the ingredients together (either simulate all the atoms of a brain, or all the molecules, or all the cells, or something higher level) and we'll get consciousness in the simulation.

Like gunpower, we won't know why these ingredients produce consciousness but we'll know they do and maybe later we'll figure out why.

[__loam]

I agree that people who think we can replicate brains on von Neumann machines with our current understanding of the brain are idiots who don't know what they're talking about, but humans build things they don't understand all the time. There's always a way to go deeper on a subject. The Romans were pretty good architects even without a modern understanding of metallurgy and structural engineering. We can treat mental illness with medications even if we don't fully understand consciousness.

[zeroonetwothree]

We can’t replicate it now but it doesn’t means it’s impossible.

Like no one thinks that humans visiting Pluto is impossible, it’s just not something we can feasibly do right now.

[thethimble]

The framework of materialism posits that there’s a physical universe and consciousness is an emergent property of physical processes. This view is so prevalent in the western world it’s hard to imagine how it could be anything else.

As an alternative, imagine that consciousness is primary. After all, any evidence that you have about the material world happens as an appearance within consciousness. (See “brain in a vat” and related thought experiments for the legitimacy of this idea).

In this alternative model, the concept of replicating consciousness with material processes doesn’t make any sense because consciousness is primary.

To be clear I’m not making any assertions about which model is correct. Instead I’m suggesting that the model you choose is axiomatic - taken as given as opposed to inferred from evidence. And starting with the latter model means artificial replication of consciousness isn’t even a logical proposition.

[fngjdflmdflg]

>I’m suggesting that the model you choose is axiomatic - taken as given as opposed to inferred from evidence

The brain is the seat of consciousness and the brain is material, therefore consciousness is emergent from material. My evidence that the brain is the seat of consciousness is that when my head hurts it impairs my thoughts, and that my eyes are connected to by brain.

Stated a bit differently:

All events must have a cause, therefore consciousness must have a cause. The brain is the most likely candidate for the cause of consciousness. The brain is material, therefore consciousness is emergent from material.

What role do you think the brain plays in consciousness? Do you believe that events must have causes?

[thethimble]

> The brain is the seat of consciousness and the brain is material, therefore consciousness is emergent from material

This is true from the standpoint of materialism but not necessarily fundamentally true.

How do you know you have a brain? As you explore this question, you’ll realize that the knowledge that you have a brain only manifests as appearances within consciousness.

It’s not necessary true that these appearances are giving you a window into an objective material universe. Instead it might be possible that your consciousness is a product of a simulation where your entire subjectivity - including the observation that you have a brain - is a manifestation of another mechanism that is outside of observability.

The point is that we simply don’t know what’s at rock bottom - an objective universe, a simulation, or an alien’s dream. Therefore the “arrow” of causality might flow from consciousness towards material as opposed to the other way around.

[fngjdflmdflg]

>it might be possible that your consciousness is a product of a simulation where your entire subjectivity - including the observation that you have a brain - is a manifestation of another mechanism that is outside of observability.

Ok. But that is equally true for any observation. For example, I don't really know that the computer I'm using to write this post actually exists under that proposition, as perhaps by brain is imagining it. So you are really rejecting observations in general here. My point is that given that observations in general are correct, then it is clear that the brain is the cause of consciousness.

[anon84873628]

>Instead it might be possible that your consciousness is a product of a simulation where your entire subjectivity - including the observation that you have a brain - is a manifestation of another mechanism that is outside of observability.

Ok, well in that simulation materialism is true and I can make an AI with emergent consciousness ¯\_(ツ)_/¯

[moi2388]

Why not? I can replicate a book in a foreign without understanding the language?

[AnimalMuppet]

You can copy an existing book, word for word (maybe not even that if it uses a different character set, unless you're doing photocopies).

Write a new book without understanding the language? No way - not one that makes any sense. Not unless you're going with the "million monkeys" approach (and if you did try that approach, you wouldn't live long enough to succeed in writing one actual coherent book in the foreign language).

So, we could think about trying to simulate a human brain, neuron by neuron. That's the "making a photocopy" approach. But that's not the approach we're pursuing. We're trying to write a new book (create a new, non-human intelligence) in a language we don't understand (that is, not knowing what intelligence/consciousness actually is).

(Side topic: What would happen if you asked GPT-4 to write a full-length novel? Or even a story as long as the token limit? Would you get anything coherent?)

[davnicwil]

As a tangent on this, it'd be such an interesting experiment to see how far one could go in deciphering/understanding a new language and attempting to write a new book in that language based on the content of a single, probably fairly long, book.

It feels like it should be theoretically possible, but I doubt it's ever been tried.

Maybe something like understanding aincent languages from limited, fragmented sources is the closest natural experiment we have in practice to have tested it, but it's hardly the same as a full, substantial text in a consistent style.

[moi2388]

Of course you can. When the brits first met the Chinese, the Chinese didn’t speak a word of English and the British not a word of Chinese.

The Chinese room thought experiment is flawed; this is exactly how we learn languages!

[hovering_nox]

Congratulations, you have just invented large language models.

[fuzzfactor]

IOW you are hardly conscious at all of what the book is about.

[mef]

without understanding the language, you still understand that it's a book, it has symbols which represent language, and those symbols can be replicated.

we don't understand what consciousness is or how it is achieved, so discussing how to recreate it isn't really a conversation we can have - we can only discuss how to create a simulacrum.

[moi2388]

That’s not because it’s some deep philosophical problem, but because people refuse to define what they mean by consciousness.

Do they mean memory? Attention? Awareness? Self awareness? The inner voice? Qualia? All of these can be explained somewhat, but they keep moving the goalpost

[mef]

I would say it's not because people refuse to define, it's that science has not defined (and cannot define it)

[jacobsimon]

I agree but even if you take materialism for granted, we’ve yet to uncover the exact biological mechanism. It’s entirely possible that it is unique to carbon-based and/or analog brains.

[zeroonetwothree]

That wouldn’t really be consistent with the laws of physics as we know them. So it would require a significant change in our scientific theories (which is possible, but I wouldn’t bet on it)

[jacobsimon]

I don’t follow you. I’m saying we haven’t discovered any inorganic consciousness, so it isn’t a given that we will be able to create it with digital computers. Not sure how that breaks the laws of physics.

[anon84873628]

"We haven't discovered any inorganic flying machines, so it isn't a given that we will be able to create them with metal."

Turns out that if you get the structure right you can mimic the effect. Mimic the brain structure in silicon and you will get a similar effect. Just like how birds and airplanes are similar but different.

[__loam]

Programmers seem to forget about how physics is the reason there's np-hard problems.

[Thiez]

Humans aren't better at solving NP-hard problems so I don't really see the connection with consciousness here.

[anon84873628]

Uh, it seems pretty clear that the "exact biological mechanism" is neural networks. Like, literally the connectedness and firing of neurons in the brain.

And no there is no proof that quantum effects play a significant role.

[naasking]

Some people are convinced by p-zombies and The Knowledge Argument that not everything we experience can be reduced to matter interactions.

[chrsw]

How is that different than saying the function of the lungs, kidneys and heart can't be reduced to matter interactions? How is the brain special?

[naasking]

You have to Google and read those thought experiments to see why. You may not be convinced (I'm not), but they give good reasons. We have mechanistic explanations for all of those organs, and even if we lack some explanation, we know one is possible in principle. They argue this isn't the case for consciousness.

[card_zero]

OK.

Philosophical zombies react to external events in exactly the same way as normal people, including internally, but we are told they lack conscious experience. Thus the thought experiment is set up from the start to find that conscious experience is something non-physical - or else the p-zombies don't really do what they're claimed to do, which is to react identically to everyone else.

There's a dubious implication that conscious experience is completely cryptic, with no effect on the outside world (such as a person speaking the words "I consciously experienced that"), or at least that all such effects are shallow enough that they can be perfectly faked. If this is true, we ought to question why it's such a big deal. What's so great about consciousness? Why associate it with rights?

The Knowledge Argument is about a scientist who learns "everything" about colors intellectually but doesn't see them until years later, and seeing a red tomato is a revelatory experience even after all that book-learning, so it implies that experiences are beyond knowledge, or beyond physics, or beyond tomatoes or something. But really all it shows is that intellectual learning is dry and dusty and limited. Like with the p-zombies, the premise is wrong. The scientist didn't really learn everything before having the experience, but could have done in principle but for the limits of communication, description, and simulation as we know those things presently. (And then the real experience would not have had any surprising or revelatory quale about it.)

[nearbuy]

> or at least that all such effects are shallow enough that they can be perfectly faked

Physicalism implies that things we wouldn't intuitively think of as conscious can perfectly mimic all such effects.

Imagine there's a person, John, and you take a precise scan of every neuron of his brain (or every particle if you prefer). You also record all the sensory input signals from his neurons to his brain. You write all this information down in a giant stack of papers. Then you go about simulating the brain with pencil and paper, computing its thoughts and actions (in this thought experiment people have deciphered exactly how neurons work). Maybe it takes you a trillion years to simulate one day of John's life, but you diligently do it.

Physicalism tells us that you can simulate John perfectly this way. You could perfectly predict every word is said, and every muscle he moved. You could feed the motor neuron outputs of your simulation into a robot replica of John, and it would act indistinguishably from the original John.

Is this pencil and paper simulation of John a p-zombie?

We either have to accept that this pencil and paper simulation of John is conscious, or that it's a p-zombie.

[anon84873628]

>Physicalism implies that things we wouldn't intuitively think of as conscious can perfectly mimic all such effects.

This is the multiple relizability argument and it only discounts certain types of physicalism that aren't popular anymore.

I consider myself a physicalism and answer that the simulation of John is conscious.

[naasking]

> Thus the thought experiment is set up from the start to find that conscious experience is something non-physical

The point is that if you accept that p-zombies are possible, then you accept that consciousness is not necessarily physical. If it's not necessarily physical, then physicalism is false.

> really all it shows is that intellectual learning is dry and dusty and limited.

What it's attempting to show is the limit of factual knowledge. If physicalism is true, then everything that can be observed must reduce to objective third person facts. But, Mary has all of the objective third person facts. So if you find it implausible that Mary would be able to infer the experience of red before actually observing a rose, even with all of those facts, then you're admitting the existence of first person subjective facts, which cannot be reduced to objective third person facts, not even in theory.

Daniel Dennett has some great responses to these challenges.

[card_zero]

Much though I'd be interested in Daniel Dennett's responses, I don't think you understood mine. I'm saying:

Mary doesn't have all of the objective third person facts, only the ones that can be conveyed to her academically.

If you want to sweep this aside with a magic gesture, and assert that she does somehow have all the facts (alright, all the objective third person facts), you are also making the science, communication, imaginative simulation, verbal learning process, all that kind of stuff, into something magical. Because what you're saying is that it now somehow has the power to be exactly like the real experience, which in this magical scenario will thus come as no revelatory surprise to her. We only expect it to be a surprise because of realism about the limits of book-learning as we know it, because she can only learn all that is explicitly known about colors that way, which is not all there is to know about them, and is not even all that is commonly known.

[int_19h]

I think OP is saying that p-zombies are a pointless thought experiment, because the intended outcome is hardcoded in the premise. If you accept the premise, you ipso facto already believe that consciousness is not necessarily physical, and the experiment doesn't change that. If you reject the premise, then the experiment is nonsensical.

[anon84873628]

>If physicalism is true, then everything that can be observed must reduce to objective third person facts.

You are erroneously equating "physicalism" with "reductive physicalism". It's clear to many of us that qualia are something subjective and non-physical yet emerge from physical processes.

[ikrenji]

well the brain is special because it gives rise to consciousness. lungs, kidneys and heart don't

[anon84873628]

The brain evolved to be intelligent. Intelligence is the ability to model the outside world in order to make predictions and then choose an optimal course of action. More intelligent beings are generally more "fit" in the evolutionary sense. Hence how humans have conquered all the ecosystems on the planet.

Turns out, consciousness/experience/qualia is an unexpected side effect of this process. We can "feel" the intelligence happening.

Where we humans get lost is believing that it has any meaning, or that we are somehow in control. We're not; the feeling of free will is just a other trick of qualia.

[nprateem]

We don't know that either. We may be a collection of interacting consciousnesses arising from different organs that all pass around the baton called "I".

[ikrenji]

we know that people don't become someone else after a heart transplant for example

[astrange]

There's a lot of nerves and bacteria in the digestive system that might contribute some of it. That and various hormonal glands and reproductive systems.

[cambaceres]

> Evolution basically says humans are conscious matter.

Evolution states nowhere that humans are concious.