[ubermon]

> Both scores come from a fixed fallback rule: Opus 4.8 and Sol xhigh run first; games scoring below 80 are rerun with Fable 5 and Sol max, respectively, and the higher per-game score is retained.

hmm, this is like pass@n until you get the high watermark? How would this mean anything?

[pixl97]

Do you throw every problem at your most expensive programmer first? Or do you toss most things to your fleet of juniors and let them knock out the easy work and pass the harder problems to your senior developers. Because that's what real world problems look like and work like in business.

Token costs are about the number one thing discussed by businesses these days.

[scotty79]

How do humans get their results at these tasks? Don't they try until they succeed and claim their best run as their result?