[roywiggins]

It feels more like an argument that chatbots will never exhibit general AI.

Which ought not to be controversial. The point of the Turing test isn't to provide a blueprint (just optimize human dialogue and you'll eventually get general AI) but a test to see that your general AI works. You might build a general AI that fails the Turing test, but you won't be able to pass the test without a general AI. That's the idea.

Unfortunately people have taken the wrong idea from the Turing Test and decided to attack the "faking human communication" thing directly. Which is fun! But anyone who in 2019 thinks that better chatbots will eventually develop general AI are delusional. I don't know if anyone with more than a passing interest actually does believe this, so it feels like this paper is arguing against a straw man.

[simonh]

Quite, the idea that mathematical modelling of language will lead to general AI is absurd. The simplest way to defeat chatbots and mathematical language generation models is to teach them something new, like a game or other rules based system and ask them questions about it and then play it. They fall flat on their face immediately because they have no ability to build, interrogate and adapt models of systems.

The authors’ credence of Searle’s Chinese Room argument is telling. The Chinese Room is misdirection. We are invited to consider an agent in a room manipulating symbols on cards and asked could such a system be considered conscious. In fact there might need to be trillions of these agents in rooms covering an area many orders of magnitude larger than the Earth, manipulating millions of trillions of symbols every millisecond. Asking if a system like that could be conscious is a whole different question.

“Here however Turing commits the fallacy of petitio principii, since he presupposes an equivalence between dialogue-ability (as established on the basis of his criterion) and possession of consciousness, which is precisely what his argument is setting out to prove.”

Sigh, no. Dialogue ability isnt claimed to be _equivalent_ to possession of consciousness, that’s putting the cart before the horse. It’s a possible product of consciousness. You could have a conscious system incapable of sensible dialogue, but the point of the test is you can’t have sensible dialogue without consciousness. That’s a claim and it’s arguable, sure, but dialogue ability doesn’t lead to consciousness. That’s daft. They and Searle look at this from entirely the wrong direction.

[roywiggins]

And everyone knows there are conscious agents who can't hold a sensible conversation: toddlers. They're more conscious than any chatbot could ever be, but they'd fail the Turing test. So would dogs, and dogs show more recognizably cognitive ability than a chatbot. Let alone nonhuman primates, who are all much, much smarter than a chatbot and would all fail the Turing test.

It would be one thing if we had built an apelike intelligence and found it impossible to make something smarter, but as we can't model them either, worrying about not entirely understanding language seems beside the point.

[ben_w]

> Unfortunately people have taken the wrong idea from the Turing Test and decided to attack the "faking human communication" thing directly. Which is fun! But anyone who in 2019 thinks that better chatbots will eventually develop general AI are delusional.

Absolutely. It’s Goodhart's law in action, targeting the behaviour that wins the prize money rather than the intelligence that the test was about. That said, I don’t think the targeting was very good last time I looked (a few years ago now), as the chatbots mostly didn’t have any long-term memory and would have conversations along the lines of “Where do you live?” “Paris” “What city do you live in?” “Newark” “What country do you live in?” “The Former Yugoslav Republic of Macedonia”. If we build something intelligent, it should pass the Turing test, but that doesn’t necessarily mean that something which passes the Turing test is intelligent.

[marcinzm]

>It feels more like an argument that chatbots will never exhibit general AI.

I feel it's even more specific, that the current approach to training and building chat bots will never exhibit general AI. Theoretically, you could make chat bots in other ways such as directly simulating a human brain down to some arbitrary level. Maybe there's things in-between as well.

[roywiggins]

Oh, yes, that's what I'm referring to as a chatbot- something built specifically to converse by modeling natural language, rather than trying to model cognition at any deeper level. Not that we can do that successfully either, but at least trying to tackle that has an honest chance of working.

This sort of feels like an updated version of Searle's argument... a chatbot is an awful lot like his 'Chinese room.'

[marcinzm]

Got it. I think the paper actually makes a reference to 'Chinese rooms.'