[IanCal]

I'd be fine if Searle just very simply said "we have a non-material soul and that's why we understand. Anything doing the exact same job but without a soul isn't understanding because understanding is limited entirely to things with souls in my definition".

> A bunch of valves, like an LLM, could mostly successfully mimic human responses,

The argument is not "mostly successfully", it's identically responding. The entire point of the chinese room is that from the outside the two things are impossible to distinguish between.

[strogonoff]

You’re talking about Cartesian mind-body dualism. It’s absolutely fine to not sneak in that view into an otherwise sound thought experiment, as it’s quite irrelevant—the concept of p-zombie from Chinese room experiment holds regardless.

> The argument is not "mostly successfully", it's identically responding.

This is a thought experiment. Thought experiments can involve things that may be impossible. For example, the Star Trek Transporter thought experiment involves an existence of a thing that instantly moves a living being: the point of the experiment is to give rise to a discussion about the nature of consciousness and identity.

Thing not possibly existing is one possible resolution of the paradox. There may be a limitation we are not aware of.

Similarly, in Searle’s experiment, the system that identically responds might never exist, just like the transporter in all likelihood cannot exist.

> The entire point of the chinese room is that from the outside the two things are impossible to distinguish between.

To a blind person, an orange and a dead mouse are impossible to distinguish between from 10 meters away. If you can’t distinguish between two things, it doesn’t mean the things are the same. Ability to understand, self-awareness and consciousness are things we currently cannot measure. You can either say “these things don’t exist” (we will disagree) or you have to say “the systems can be different”.

[IanCal]

You seem confused as to what I’ve said. I know these things cannot exist in reality.

The Chinese room is setup so that you cannot tell the difference from the outside. That’s the point of it.

> If you can’t distinguish between two things, it doesn’t mean the things are the same.

But it does mean that the differences between them are irrelevant to you by definition.

> Ability to understand, self-awareness and consciousness are things we currently cannot measure. You can either say “these things don’t exist”

Unless you have a way they could be measured but we just lack the technology or skill then your definitions are of things that may as well not exist because you cannot define them. They are vague words you use and are fine if you accept you have three major categories “yes and here’s why, no and here’s why and no idea” that’s fine. I am happy saying I’m conscious and the pillow next to me is not. I don’t have a definition clear enough to say yes/no if the pillow was arguing with me.