[codeulike]

Turing test was definitely and conclusively refuted in the 1960s

Are you sure?

Understood properly, Turings Imitation game aka the turing test, should be adversarial. That is, the player should be asking hard questions to try and discover who is who, not just having an idle chat. No chatbot has been able to consistently pass an adversarial Turing Test until the rise of LLMs

The Imitation Game:

https://www.cs.ox.ac.uk/activities/ieg/e-library/sources/t_a...

[LeCompteSftware]

The fact that LLMs often score as "more human" than actual humans is a downstream consequence of ELIZA tricking people into thinking it had a glimmer of consciousness. The Turing test was refuted because it was proven scientifically meaningless in the 1960s, and LLMs only reinforce that.

[dpark]

What refutation are you referring to? Surely you can cite how it was "proven scientifically meaningless" some 6 decades ago.

[LeCompteSftware]

I did with the Weizenbaum link, but here's a specific refutation: https://en.wikipedia.org/wiki/ELIZA_effect

The Turing Test is totally meaningless, as was conclusively demonstrated some 6 decades ago. It is a test that measures how well your program can fool humans, which means "intelligence of the computer" is hopelessly conflated with "social engineering chops of the humans who programmed the computer." Any computer scientist who takes it seriously should be deeply embarrassed because they are spouting sci-fi adjacent nonsense, not actual science. Actual science involves updating your priors based on evidence.

[dpark]

I must have missed your earlier link.

Eliza did not disprove the Turing test, though. What it showed is that it’s easy to pass a very scoped test that doesn’t allow the user to actually broach general topics. Anyone communicating with Eliza for general conversation should quickly discover that it’s running a script. Just “What time is it?” breaks the script.

The Turing test was never that a computer could convincingly simulate a human convincingly in a very narrowly scoped scenario. Certainly the Eliza effect is interesting because it shows that people can assume emotion where it doesn’t exist (they do the same for other humans, by the way), but it does not disprove the Turing test.

> Any computer scientist who takes it seriously should be deeply embarrassed because they are spouting sci-fi adjacent nonsense, not actual science.

This sentiment seems to be expressed a lot from people who want to insist machines can’t be conscious. This retreat to shaming those who disagree is a tacit admission of a weak position.

Convince with logic and facts if you have them.

[cv5005]

Yeah I dont think a single current LLM would fool me in a turing test - I would obiously use all kinds of prompt injection techniques, ask about 'dangerous' or controversial topics, ask about random niche facts in varied fields, etc.

[codeulike]

Thats a good point actually, I hadn't thought about deploying hacks and jailbreaks in a turing test but thats exactly what should be done, if its being done adversarially