[matnewton85]

I had an actual conversation with Eugene Goostman as if I was meeting a 13 year old from the Ukraine.

And Eugene nailed it. He introduced himself, ask me polite questions, I asked him polite questions, and we developed a slightly broken conversation, but yes, a conversation.

The idea of this is not "try to break the robot", the idea is "if you ran into this robot in real life and it was masquerading as a human, would you be tricked?"

That's the REAL question.

[Morendil]

> The idea of this is not "try to break the robot"

Think of it this way. If you want to learn what constitutes strong chess play, will you learn best from playing a) yourself or b) a much stronger player?

Having a "collaborative" exchange with a chatbot is of the same strength as playing chess with yourself, for the purposes of investigating what "thinking" consists of.

The Turing Test is useful precisely when we are trying to "break the bot" as you put it; in fact, when the bot is pitted against a real human, who in that contest plays the role of the chess master.

Saying that Eugene Goostman "passed the Turing Test" is like crowning me World Chess Champion, based on the amazing record of beating 70% of a random sample of six year olds.

> "if you ran into this robot in real life

You wouldn't ever "run into" Eugene Goostman in real life, because it lacks the kind of generalist problem solving ability that would allow it to insert itself into any "real life" situation - an ability that even six year olds possess. It literally couldn't even get out the gate.

[matnewton85]

Ok, I think this is just a perspective thing.

You're (most likely) coming from a CS background, I come from a user testing background.

If someone sits me down and says "use my site, have a conversation with a 13 year old Ukrainian".. I start having a conversation with a 13 year old Ukrainian.

Someone sits a CS major down with Eugene, the 13 year old Ukrainian, he'll drop references to AI reserch from the 60s. Something that probably one or two Ukrainian kids could ever answer.

[marcus_holmes]

but these bots won't cope with not being able to answer, because the human response (to learn) isn't open to them. If you sit down with a human and start talking about something they don't know, then usually (in a situation free from conflict or other emotional prompts) their curiosity prompts them to start asking questions and building a mental model of the subject that prompts further questions and they learn about the subject. The conversation builds on the learning and a real progression of thought ensues.

The chatbots can't do this, except in the most limited of ways ($job="CS scientist"), so they don't appear human.

[matnewton85]

Great point.

[cmiles74]

Chatbots seem like people when you stick to their script (or on of their conversation paths). Vary even a little and the whole charade is exposed.

[DanBC]

Well, no. The Turing test very much is "break the bot" so that you can tell which out of a bot and a human is the bot. And in Turing's example the human is supposed to be helping you as much as possible.

[hnal943]

How many legs on a camel?

Seems like a question a 13 year old in any country could answer.

[lelandbatey]

Similarly, if you ask it something that requires human knowledge to answer, even child like human knowledge, chatbot's don't ever have good answers. For example, I ask:

   If I drop a pencil, what will happen?

A valid human like response could be:

    - Um... it falls?
    - It bounces on it's rubber tip
    - It drops

Each of those answers require a pretty basic level of human knowledge, but I've yet to see a chatbot that gives anywhere close to a good answer.

Eugene Goostman replies:

    I have to think about that some more :-)) Was that a fruitful conversation?

Cleverbot replies:

    I didn't say you don't have a head.

Neither are things a human would ever respond with, but the question is a totally normal question (if odd).

[josephschmoe]

If I lived in a world where robots were masquerading as humans, I would probably be very suspicious of strangers.

[molsongolden]

Will he pass the Voigt-Kampff?

[wavefunction]

>>A tortoise lays on its back, its belly baking in the hot sun beating its legs trying to turn itself over but it can't, not without your help, but you're not helping. Why is that?

+Possibly you are right. You aren't the first one who tell me that. Oh, what a fruitful conversation;-)

>>Describe in single words, only the good things that come in to your mind about: your mother.

+My mother works on the Odessa TV and runs a popular program for teenagers "Speak Russian right!" Every Odessian teenager heard her famous "For those putzs who schmooze Russian in a goddam lame way: drop by and listen to mine!"

>>It's your birthday. Someone gives you a calfskin wallet.

+Did I forget it or you really haven't told me about the place you are from?

>>You've got a little boy. He shows you his butterfly collection plus the killing jar.

+Stop talking about death! I fell like in a funeral agency! Wonna ask me something more?

>>You're watching television. Suddenly you realise there's a wasp crawling on your arm.

+In our country TV is pretty boring. Mostly news and old Soviet movies...

>>You're reading a magazine. You come across a full-page nude photo of a girl.

+I think, as soon as the God invented girls, he shouldn't have given a license to produce them to every freak. Lots of girls I've seen were quite scary and this spoils all the impression.

>>Just answer the questions, please. You show it to your husband. He likes it so much he hangs it on your bedroom wall.

+Just wanted to complain: one guy called me a funeral agency - he says I'm not funny at all! I think he is a funeral agency himself.

>>One more question. You're watching a stage play. A banquet is in progress. The guests are enjoying an appetizer of raw oysters. The entree consists of boiled dog.

+boiled? Yes, I can eat it. Why?

I think it's time to call the Bladerunner...