Hacker News new | ask | show | jobs
by YurgenJurgensen 895 days ago
In actuality, it primarily tests the knowledge of the user, not their intelligence. I'm sure that GPT-written paragraphs seemed really impressive when you first encountered them, but nowadays half the Internet has seen enough of them to recognise the default ChatGPT style in less than the space of a Tweet. People aren't significantly smarter than they were a few years ago, but I bet GPT-3.0 will perform significantly worse on a Turing test now than it did the day it was released.

Similarly, I believe a lot of early Turing Test successes kind-of cheated and had their bots pretend to be ESL, on the grounds that the interrogators would interpret their unnatural responses not as a robot but as a second-language speaker's human mistakes. But people who teach English as a second language, or interact with language-learners a lot will learn the types of mistakes each group of learners make, and will spot unnatural mistakes a lot faster.

Now that I think about it, that a major factor in determining Turing test performance isn't the intelligence of the testers but their knowledge does highlight why it's not a great measure of intelligence in the first place.