[_xnmw]

I'm with John Searle on the Chinese room [1] opinion, i.e. that a machine cannot be said to "understand" language even if it is able to pass the Turing Test. That is because when we say "understand", we are referring to particular kind of human experience (qualia?) that a machine simply doesn't seem to have, but animals, for example, do.

[1] https://en.wikipedia.org/wiki/Chinese_room

[goto11]

Unfortunately you have no way of determining if a machine or animal have this particular experience qualia. You cant even determine of other people beside yourself have it, which gives rise to solipsism.

It like saying that red-headed people doesn't have a soul - there is no way to disprove that assertion.

[_xnmw]

The point is not whether we can prove that machines have such an experience. The point is, most people will agree that it “seems” like machines don’t have feelings, which is itself interesting because it suggests that our intuitive definition of “understanding” is not limited to a logical set of inputs and outputs. It is a proof that when we say “he understands”, we are referring to something more than a logical answer, and whatever that “something more” is, machines don’t have it.

[goto11]

In puppet theater good puppeteers make the figures look like they think and feel very convincingly, even though we know they are just made out of wood. I'm sure the same can be achieved with robots.

[jplayer01]

That's the fundamental issue underlying the discussion. Pointing this out doesn't add anything. It's like saying water is wet. Yes, but how do we quantify human understanding such that we can implement it in AI? Well, we've been puzzling over that for decades and we sill haven't found an answer.

[msla]

I can say that you don't have qualia and you can't prove me wrong.

Does that seem dangerous to anyone else?

I also don't see any distinction between "qualia" and "soul" other than spelling, but perhaps it's because I don't have one.

Finally, I have this question for Searle: Say you understand English. Does any specific neuron in your brain understand English? No, the larger system of neurons+neuronal connections does, so why doesn't the system of grad student+book understand Chinese?

[jplayer01]

I'm not sure where you're going with this. Are you implying that thinking about this problem is dangerous in itself? That would be absurd. We've been philosophizing about qualia or what it means to be human or whether we can even be certain anybody else shares the same experience of reality or even if they're human at all for centuries. I mean, what else does Descartes do in the process of his discourse? I don't think that's caused undue suffering in the same way that organized religion has.

All it shows is that after hundreds of years, we still don't know how to explain or quantify human consciousness.

[baddox]

Dangerous? No. To me it just seems to mean that “qualia” is not a particularly useful concept, particularly when discussing the capabilities of computer software.

[msla]

"Dangerous" as in "Oh, you know they don't have qualia, so we can do whatever we want to them. They don't really feel pain, they just act like they do."

[goatlover]

The issue is that we’re using a word based on human mental activity and social agreement and then applying that to a computational process in a machine, which likely leaves out part of the human experience which makes up the word understand.

It’s more accurate to say the Chinese room computes results which humans recognize as successful translation from English to Chinese. The understanding is all on the side interpreting the output.

[tshaddox]

Then what is the point in asking if a machine “understands” English and Chinese? It sounds like the question would be either completely untestable, or the answer would just be “no” by definition because we’re defining “understanding” to be “based on human mental activity.” It just doesn’t seem like a useful question if the answer can not be determined by a test such as the Chinese room thought experiment.

[goatlover]

Well, if we asked this about Data from Star Trek, then the answer would have to be yes, or mostly yes (Data does struggle to make sense of some human behavior on the show). So then the question is what gives Data an understanding that the Chinese Room lacks?

Data participates in human society and he has a human-like body. Data also has subjective experiences, as evidence by his dream sequences in one episode. Whereas the Chinese Room is just following a bunch of rules for translation. But Data doesn't merely translate from one set of symbols to another given a large set of rules. He learns by interacting with people and his experiences as an android. From that we could say understanding is the result of an embodied social activity that the Chinese Room completely lacks. Whatever the Chinese room is said to be doing, that's not the same as understanding language.

Another way to put it is that language isn't equivalent to symbol manipulation, even though it makes use of symbols, or a least since the written word was invented.

[_xnmw]

The point of asking such a question is to prove whether or not computational processes accurately model the human brain.

Unfortunately this is a recursive question, because the only device we have for exploring the difference between a brain and a computer is our brains. Thus, I believe the Chinese room experiment is rightly composed of as a thought experiment - what other means do we have for assessing our difference from computers other than our intuition?

[goatlover]

Do you have color, sound, taste experiences? Do you experience pain? Do you dream or imagine? Then you have quaila. If you don’t like the philosophical connotations of that term, then call it subjective experience.

[msla]

> Do you have color, sound, taste experiences? Do you experience pain? Do you dream or imagine?

I can say I do, but what reason do you have to believe me?

[goatlover]

It's a reasonable inference given your similar biology and behavior. Doubting your subjectivity is on the level of doubting that anything exists outside my experience, or doubting that the universe existed more than five minutes ago. I can't prove those things are true, but I have no good reason to have such doubts.

Still, it remains a philosophical problem, even more so for animals or robots like Data. That's what Ned Block called The Harder Problem of Consciousness. But here I think we just have to accept that our knowledge of others and the world lacks certainty. We trust our senses and inferences to form a reasonable view of the world, but we can never be sure.

[friendlybus]

Dangerous? Only as much as you want to swim in nihilism. Someone who claims you have qualia, a soul and a psychic commitment to astrology or whatever is more dangerous because that person has an affirmative commitment to maintaining that belief.

Qualia is generally argued by Sam Harris to be simple or reductionist elements of our human experience we can all agree humans share. Burning your finger on a hot stove and recoiling is a conscious experience every human shares.

The soul includes way more ideas and depends on who you talk to. The word has been overloaded a bunch, but generally can be said to include a higher spiritual aspect.

[baddox]

It’s interesting that you said the machine doesn’t seem to have the experience, but animals do seem to have it. Can you explain what you mean by that?

[_xnmw]

It’s a feeling. Understanding is a feeling, not just a logical set of inputs and outputs.

[js8]

See my other comment in this discussion. What you experience as "understanding" is that a particular constraint satisfaction problem (roughly, the logical fragment that is supposed to be understood) has a solution and you are able to construct it.

I have also somewhat responded before to Chinese room argument with this comment: https://news.ycombinator.com/item?id=20864005