[bevekspldnw]

Wish it had GPT-4, that’s the one to beat still.

[GuB-42]

It is there, not for all the benchmarks, but for those where it is included, GPT-4 scores much higher.

Not surprising since GPT-4 is still state-of-the-art and much bigger. Where Mistral has been particularly impressive is when you take the size of the model into account.

[mirekrusin]

GPT-4 is instruct tuned model, of course it's going to score higher, apples and oranges.

[bevekspldnw]

Yeah and the instruct tunes provided by Mistral on other models are pretty great.