[mirekrusin]

Not to take anything from what you're saying but any person involved in hiring process _for programmers_ will agree that people struggle with trivia far easier than this example.

[samrus]

Yeah but an LLM can't be excused being nervous or not tired or just not operating at its best like a human can. They are always operating at their best, so if they trip up once, that's their limit

[Terretta]

> An LLM can't be excused... LLMs are always operating at their best

Depends how much you turn down the heat.

[mirekrusin]

Yes, but they will only keep getting better; whether we like it or not sooner or later we’re going to be hiring api keys, not people.

[k__]

"they will only keep getting better"

I don't think that's a given.

It could very well be that they only get smaller, faster, or even worse when they accidentally start training them with hallucinations that spread on the web.