[kec]

Imprecise language, but the article is specifically referring to questions like “why do you think I asked you that?” Or “how are you answering these questions?”. LLM’s can’t engage with these types of questions, the best they can do is to regurgitate a canned response peppered with some prompt history.

[kenjackson]

In fairness most humans can’t either. Try going to a random person at the park and asking them “Explain the relationship between Romeo and Juliet and Star Trek”. And then ask them why they think you asked that question. They’ll mostly be befuddled I suspect.

[robbomacrae]

I had to go and try this exact line of questioning with ChatGPT because I suspected this might lead to a weakness in it not admitting when it just doesn't have a clue (which would have been my answer)... mind you its a big human weakness/tendency to not admit lack of knowledge.

But the answer was surprisingly candid and yet thoughtful:

""" I can't know for sure why you asked the question about the relationship between "Romeo and Juliet" and "Star Trek," as I don't have access to your personal thoughts or context. However, some potential reasons might include:

Academic Inquiry: You might be exploring themes in literature or media studies and are interested in drawing connections between different works across genres and time periods. Creative Inspiration: If you're a writer, artist, or content creator... """

There were some others but overall I thought the initial disclaimer along with some possible theories approach was spot on and a lot better than my "no clue" knee jerk reaction.

[almost_usual]

So knowledge or memorization of culture is intelligence?

What if that personal steals your wallet without you being aware while you ask them that question because they need food. Is that intelligent?

[stavros]

That's not what the GP is saying. The claim is that the inability to answer about culture isn't a sign of a lack of intelligence.

[post-it]

They didn't say anything about intelligence. I think you might be parsing this thread differently than intended.