[rvz]

You are thinking too hard on this. This entire "benchmark" is a performative joke for attention that only works on HN.

> What will we have in five years? Maybe it will all have puttered out, and we will have moved to the next thing?

We will just have more of the same.

[simonw]

> This entire "benchmark" is a performative joke for attention that only works on HN.

I take exception to that! It's a performative joke for attention that works far more widely than just Hacker News.

[Yiin]

You say it's performative joke, but it all depends what you're using model for. So far the rule has been quite straightforward, better models consistently renders pelican in higher quality, I've yet to see an exception. It is also a good enough (for me at least) test for "taste" the model has.

[j_maffe]

> better models consistently renders pelican in higher quality The article literally avoid making this argument and gives counterexamples to this statement.