[rtlacn]

Humans can solve these riddles even if they have never seen a similar one.

LLMs need to have seen at least a similar riddle with the same structure and/or code to plagiarize.

Humans can deal with modified riddles if they are alert and awake. An LLM is always alert and awake, so it should be able to handle modified riddles, but it cannot.

[mirekrusin]

Not to take anything from what you're saying but any person involved in hiring process _for programmers_ will agree that people struggle with trivia far easier than this example.

[samrus]

Yeah but an LLM can't be excused being nervous or not tired or just not operating at its best like a human can. They are always operating at their best, so if they trip up once, that's their limit

[Terretta]

> An LLM can't be excused... LLMs are always operating at their best

Depends how much you turn down the heat.

[mirekrusin]

Yes, but they will only keep getting better; whether we like it or not sooner or later we’re going to be hiring api keys, not people.

[k__]

"they will only keep getting better"

I don't think that's a given.

It could very well be that they only get smaller, faster, or even worse when they accidentally start training them with hallucinations that spread on the web.