[OutOfHere]

Did Anthropic give you third-party benchmarks? Is that what you said to them? Yes, they're important, but the attitude is wrong.

[bloppe]

Anthropic always publishes 3p benchmarks every time they announce a new model

[MostlyStable]

And even if they didn't, they have a track record. Even if we did have benchmarks in this case I would still wait until people got there hands on it and formed a more holistic opinion.