[TrainedMonkey]

Worry not, Pelicans on bicycles had been ranking pretty high on your favorite search engine for a while. I struggle to imagine a world in which it was not already scraped and turned into training data by at least one provider:

1. Models need to be good at the questions we ask them, not the questions we could ask them.

2. The questions, at least partially, are correlated with information people consume.

3. People mostly consume viral content.

4. Ergo you should scrape viral content for training data.

[ruszki]

Probably it was added to the training data on the first day when this benchmark was on HN main page. It’s a bad benchmark since then. I don’t know why people still rate it high. Basically, every benchmark becomes pointless after it was published. They are good only to have a picture at the time they’re published first, and not after.

[simonw]

If it was added to the training data on day one, why can't any of the models draw a decent picture of a pelican riding a bicycle?

[ruszki]

Most of them were completely unrecognizable back then. Compared to those, this is a huge achievement.