[necovek]

Sounds like one peculiar benchmark that a particular model could be trained on.

Btw, how do you get 17-20x? Cost seems to be 0.07 to 0.55 (for 4.6, or 8x) and 0.75 (for 4.5 or 11x).

[thelinuxkid]

The 17-20x is output token pricing: MiniMax M2.5 Standard at $1.20/M output vs Opus 4.6 at $25/M = ~21x. On input it's $0.30 vs $5.00 = ~17x. Blended (3:1 input:output) works out to roughly 19x. Curious where you're getting 0.07 and 0.55 — are you looking at M2.5 Lightning pricing or a different provider?

[necovek]

I look at "Avg $" (average task cost per instance) from the page you linked to.