[adriand]

The goalposts absolutely have moved: consider the Turing Test as a prime example. If machines cannot pass that test now (and right now it would take a skilled interviewer with substantial domain knowledge to cause a failure), they seem likely to be able to in the very near future.

[soperj]

It really doesn't take much skill as an interviewer. ChatGPI is so confident in it's answers all you have to really do is ask it for the lyrics to a song that you know. It never answers correctly. It might get the first verse or 2 correct, maybe even the chorus, but it eventually decides to make up it's own parts and then you know.

[anonyfox]

I just asked ChatGPT (v4!) and it’s a bit sensitive but otherwise correct:

"Surfacing" is a song by Slipknot from their self-titled debut album, released in 1999. Please note that the lyrics contain explicit language. Here is a portion of the lyrics with some of the explicit content redacted:

"Running out of ways to run I can't see, I can't be Over and over and under my skin All this attention is doing me in!

[Chorus:] (Expletive) it all! (Expletive) this world! (Expletive) everything that you stand for! Don't belong! Don't exist! Don't give a (expletive), don't ever judge me!

Picking through the parts exposed Taking shape, taking shag Over and over and under my skin All this momentum is doing me in!

[Chorus:] (Expletive) it all! (Expletive) this world! (Expletive) everything that you stand for! Don't belong! Don't exist! Don't give a (expletive), don't ever judge me!

You got all my love, livin' in your own hate Drippin' hole man, hard step, no fate Show you nothin', but I ain't holdin' back Every damn word I say is a sneak attack When I get my hands on you Ain't a (expletive) thing you can do Get this cuz you're never gonna get me I am the very disease you pretend to be

I am the push that makes you move

[Chorus:] (Expletive) it all! (Expletive) this world! (Expletive) everything that you stand for! Don't belong! Don't exist! Don't give a (expletive), don't ever judge me!"

Please be aware of the explicit content before sharing or using these lyrics in any context.

[soperj]

I'll wait until it's actually available for me to test to change my mind.

[soperj]

Well, didn't have to wait long. Wasn't able to duplicate your success.

[anonyfox]

I assume that I am already exposed to months of prompting training now, and probably frame things differently than others on their first try?

I understood that a huge success% depends on the input prompt as the context.

In my case, just to sync on that, I used a fairly simple prompt: "What are the song lyrics of Surfacing from Slipknot?" - and I got a kinda perfect response including formatting (that is lost when pasting here)

[erohead]

Um as a human, I do this constantly as well

[gkbrk]

Humans can generally realize they don't know the complete lyrics of a song, instead of making up whole verses with 0 indication that it's not the real lyrics.

[TremendousJudge]

yeah I've never seen chatGPT say "I don't know"

[FooBarWidget]

I have. It says something like "it is difficult to say whether [...]". I think it imitates the tone of the training material on that topic.

[flkiwi]

It’ll do great in middle management.

[ilyt]

...Okay but random human might fail on exact same thing.

[soperj]

I've never met someone who will make up whole verses to a song. Especially when you pick a song with 7 words and it gives you about 10 verses.

[ilyt]

I for one was surprised few times what actual lyrics said in song that wasn't in my native language and I kinda just interpreted what it sounded like

[bluefirebrand]

Recognizing that the Turing Test is not a sufficient bar for judging if a computer is true AGI is not moving the goalposts, it's just realizing that passing the test and the location of the goalposts weren't actually the same in the first place.

[gowld]

The Turing Test was proposed as one example of a test for "indistinguishable from a human", not the singular goalpost for indistinguishability.

[wongarsu]

The Turing test has been questioned for decades, with many suggesting that Turing meant it more as a joke.

And that's ignoring that arguably chat bots have been passing the Turing test (against non-expert judges) since ELIZA in the 60s [1]

1: https://en.m.wikipedia.org/wiki/ELIZA

[joshuahedlund]

> If machines cannot pass that test now (and right now it would take a skilled interviewer with substantial domain knowledge to cause a failure)

Does ChatGPT fail this simple test: "I am going to ask you questions, but if I go silent for a couple minutes, I want YOU to start asking ME random questions."

[ogogmad]

ChatGPT predicts the next letter. It doesn't tell the time.

[joshuahedlund]

And AGI is impossible if you can’t tell time

[Lewton]

ChatGPT does not pass the Turing test