[cjtrowbridge]

I love how they include a helpful chart that shows this model scores worse than everything else.

[kibibu]

Am I looking at the wrong table? It dominates everything on visual interpretation benchmarks.

Edit: specifically ocrbench and VQAv2

[Der_Einzige]

It's not that bad, and I'd much rather that they be honest instead of lying like everyone else does.

[butterfly42069]

All jokes aside (and that did make me laugh) at least they're not training just to hit the benchmarks, which seem to be more meaningless as a quality indicator with each passing day.

[miffy900]

I see at a few models (3 models in MMMU) that score lower than Nvidia's. But putting that aside, they at least get points for apparent objectivity. At least they probably aren't fudging numbers.

[GaggiX]

Well but it actually doesn't, unless you're looking only at MMMU.

[dr_kiszonka]

Exactly. On some benchmarks it is close to or better than GPT 4o.

I wonder if one of the reasons they released it was to respond to OpenAI's plans to enter the chipmaking market.