[ZeljkoS]

Here is the unofficial benchmark: https://huggingface.co/mistral-community/Mixtral-8x22B-v0.1/...

[bevekspldnw]

Wish it had GPT-4, that’s the one to beat still.

[GuB-42]

It is there, not for all the benchmarks, but for those where it is included, GPT-4 scores much higher.

Not surprising since GPT-4 is still state-of-the-art and much bigger. Where Mistral has been particularly impressive is when you take the size of the model into account.

[mirekrusin]

GPT-4 is instruct tuned model, of course it's going to score higher, apples and oranges.

[bevekspldnw]

Yeah and the instruct tunes provided by Mistral on other models are pretty great.