[holtkam2]

at a certain point you're gonna need to change your benchmark because this will end up in the model's training set

[simonw]

Gemini were the team most likely to have this in their training set - see https://x.com/JeffDean/status/2024525132266688757 - and yet their latest model still messes up the bicycle frame!

[recursive]

I'm sure that certain point came and went many releases ago.

[kzrdude]

As mentioned in another recent thread, that time is now.