[soperj]

It really doesn't take much skill as an interviewer. ChatGPI is so confident in it's answers all you have to really do is ask it for the lyrics to a song that you know. It never answers correctly. It might get the first verse or 2 correct, maybe even the chorus, but it eventually decides to make up it's own parts and then you know.

[anonyfox]

I just asked ChatGPT (v4!) and it’s a bit sensitive but otherwise correct:

"Surfacing" is a song by Slipknot from their self-titled debut album, released in 1999. Please note that the lyrics contain explicit language. Here is a portion of the lyrics with some of the explicit content redacted:

"Running out of ways to run I can't see, I can't be Over and over and under my skin All this attention is doing me in!

[Chorus:] (Expletive) it all! (Expletive) this world! (Expletive) everything that you stand for! Don't belong! Don't exist! Don't give a (expletive), don't ever judge me!

Picking through the parts exposed Taking shape, taking shag Over and over and under my skin All this momentum is doing me in!

[Chorus:] (Expletive) it all! (Expletive) this world! (Expletive) everything that you stand for! Don't belong! Don't exist! Don't give a (expletive), don't ever judge me!

You got all my love, livin' in your own hate Drippin' hole man, hard step, no fate Show you nothin', but I ain't holdin' back Every damn word I say is a sneak attack When I get my hands on you Ain't a (expletive) thing you can do Get this cuz you're never gonna get me I am the very disease you pretend to be

I am the push that makes you move

[Chorus:] (Expletive) it all! (Expletive) this world! (Expletive) everything that you stand for! Don't belong! Don't exist! Don't give a (expletive), don't ever judge me!"

Please be aware of the explicit content before sharing or using these lyrics in any context.

[soperj]

I'll wait until it's actually available for me to test to change my mind.

[soperj]

Well, didn't have to wait long. Wasn't able to duplicate your success.

[anonyfox]

I assume that I am already exposed to months of prompting training now, and probably frame things differently than others on their first try?

I understood that a huge success% depends on the input prompt as the context.

In my case, just to sync on that, I used a fairly simple prompt: "What are the song lyrics of Surfacing from Slipknot?" - and I got a kinda perfect response including formatting (that is lost when pasting here)

[erohead]

Um as a human, I do this constantly as well

[gkbrk]

Humans can generally realize they don't know the complete lyrics of a song, instead of making up whole verses with 0 indication that it's not the real lyrics.

[TremendousJudge]

yeah I've never seen chatGPT say "I don't know"

[FooBarWidget]

I have. It says something like "it is difficult to say whether [...]". I think it imitates the tone of the training material on that topic.

[flkiwi]

It’ll do great in middle management.

[ilyt]

...Okay but random human might fail on exact same thing.

[soperj]

I've never met someone who will make up whole verses to a song. Especially when you pick a song with 7 words and it gives you about 10 verses.

[ilyt]

I for one was surprised few times what actual lyrics said in song that wasn't in my native language and I kinda just interpreted what it sounded like