They're comparing against gpt-4-0125-preview, which was released at the end of January 2024. So they really are beating the market leader for this test.
What matters here is that what I can use today. I can either use Claude 3 or GPT 4. If the Claude is better, it is best on the market. Let’s see what the story is tomorrow.
Go ahead, no one is saying to stay with GPT4. But its disingenuous to compare a gpt-4-march-update to a completely new pretrained model like Claude 3 Opus.
It is not that disingenuous. We can only make claims based on the current data.
There can be even bigger competitors in the market, but because they stay quiet and do not publish results, we do not know about their capabilities. Who knows what Apple has been doing all this time? They sure have capabilities. Even if they make some random comments about the use of Gemini.
Until the data and proof has been provided, it is accurate to claim "the best model on the market". Everything else is hypothetical.
So you think whatever process produces a GPT4 update is completely equivalent to pretraining and RLHF'ing a brand new model with new architecture, more data, etc??
GPT5 will be substantially better than even the latest GPT4 update.