[aspenmartin]

> Im afraid that the usual mantra that "we just need more scale" that worked well for attracting investments, is not working anymore - bigger models provide marginal improvements while naturally get much more expensive to run.

It's super interesting to hear this refrain on HN, it is alarmingly common. Anthropic released benchmark numbers on Mythos, as they have for all of their models. Once models become public, people evaluate them in a myriad of ways. We have had reliable scaling laws for years and they still hold. Epoch capability index continues to grow exactly as expected. Where does this idea come from?

As for cost, the cost per token at a given level of performance drops up to 40x per year.

[cmxch]

Mythos numbers are effectively irreproducible aside from cherry-picked approvals.

[aspenmartin]

Yes absolutely. However the benchmarks they did release numbers for are key ones and the leaps were very large. Absolutely possible that they either lied or that the full picture is much muddier but based on the numbers they do show that’s hard for me to imagine a likely scenario that would produce that.