[Ukv]

> Your analogy inadvertently once again emphasizes the point. Now change it from being a rabbit and a turtle, to an unknown animal and some non-animal thing pretending to be another unknown animal. And you have to guess which is which. It would be effectively impossible to figure anything out, because you have absolutely no basis to work from.

It'd be possible to get an idea if there was some box movement that was unique to animals. That's not particularly interesting because it's fairly uncontroversial that a box-sized robot could very accurately imitate an animal through the medium of box movement, but for a bot to imitate a human through the medium of text (seen as a sufficiently general interface to test "almost any one of the fields of human endeavour that we wish to include") is interesting to many.

But, the concept the analogy was demonstrating was really just basic reasoning. That, if you're given X xor Y and have evidence of Y, you should tend towards Y even lacking direct evidence for/against X. Do you agree that, in my example, you would choose the box giving some evidence of being a rabbit over the one that gives none?

> LLMs are trained on nothing except the corpus of human knowledge. It is literally impossible for them to e.g. accidentally say something that it's inconceivable for a human to say

Depends on what you mean by "inconceivable", but it's certainly possible for it to say things that it is unlikely for a human to say due to the bot's limitations (at the extreme, consider a Markov chain). And, even if only saying things that a human could just as well say, if those things are also trivial for a bot to say it is poor evidence of personhood.

> And no, always giving bad answers it not a failing strategy. As I mentioned, the scenario I'm describing is not a hypothetical. The Turing Test (or at least yet another abysmal bastardization of it) [...]

To put relevant emphasis on my claims:

> > Always giving bad answers just because humans can also give a bad answer is already a failing strategy with low success rate when the test is carried out as Turing specified

> > Then the real human B would, on average, offer far more compelling evidence of personhood and the bot would fail the majority of the time. I don't see how this issue affects Turing's proposed version of the experiment.

I agree that there are ways to bastardize the test. If for instance you have no second player that you must choose between (have to say A xor B is a bot), then just remaining silent/incoherent to give no information either way can be a reasonable strategy. As with all benchmarks, you also need a sufficient number of repeats such that your margin of error is low enough - fooling a handful of judges does not give a good approximation of the bot's actual rate.

I'd even claim it's a bit of a bastardization to use Turing's 30% prediction (of where we'd be by 2000) to reduce the experiment down to just pass or fail. Ultimately the test gives a metric for which the human benchmark is 50%.

[somenameforme]

Wellp this was a fun conversation, but it seems to me that at this point there's not much more to do other than repeat ourselves. The final thing I'd emphasize is that it's important to make sure metrics measure what you want them to measure. To some degree we've already ruined the name of the Turing Test with excessive simplifications. 'Oh that? Yeah, it was passed a decade ago, right?'

Of course one practical issue that, in some ways, makes this all moot - is that if we ever create genuine AI systems capable of actual thought, the entire idea of a "test" would be quite pointless. Rapid recursive self improvement, perfect memory, perfect calculation, and the ability to think? We'd likely see rapid exponential discovery and advancement in basically ever field of human endeavor simultaneously. It'd be akin to carrying out a 'flying test' after we landed on the Moon.

[Ukv]

I think we generally both agree that there are some poor misimplementations of the test, like the one you linked where (according to their paper) the interrogator could answer "unsure" on a bot's response and count as being "fooled" by that bot even if they then answer "human" on a human, which does allow for giving nonsense answers to be a legitimate strategy (unlike with Turing's specification, I'd claim).

Ultimately I do think Turing's experiment measures something interesting. There's a nice "minimal maximality" to it, in that it's a simple game yet set up in a way that solving it encompasses all facets of intelligence that current humans have. Maybe coincidentally comparable to the test for Turing completeness, in that a Turing machine is conceptually simple yet simulating it proves computational universality. I feel there's a risk of missing the nuance and just taking the experiment as a singular benchmark, whether it's made "easier" or "harder", akin to "simulating a Turing machine is too easy, how about simulating the Numerical Wind Tunnel?"

> Rapid recursive self improvement

I'm a bit sceptical of a hard take-off scenario.

Say on first pass it cleans up a lot of obvious inefficiencies and improves itself by 50%. On the next pass it has more capacity to work with, but the low-hanging fruit are already dealt with, so it probably only manages to squeeze out an extra 10%. To avoid diminishing returns, it'd need to automatically build better chip fabrication plants, improve mining equipment, etc. so that many steps in the pipeline are improving. This will all happen eventually, and contribute to humanity's continuing exponential progress, but IMO will be a relatively gradual changeover (as is happening now) rather than an overnight explosion from some researcher making a bot that can rewrite itself as soon as it can "actually think", whatever that entails.