Price of the current frontier may vary, but price for a given level of capability tends to drop pretty fast.
April of last year you'd get 1431 ELO[0] from o3-2025-04-16 for $8.00 per million output tokens. April of this year you can get 1436 ELO from deepseek-v4-flash for $0.2 per million output tokens.
Sure, but i don't think it's reasonable to hold given level of capability constant in a landscape where a give consumer of AI also has competitive pressures.
I can't use last year's SOTA model when my competitors can use the current SOTA model.
This is also baked in the eye watering valuations of model companies.
> I can't use last year's SOTA model when my competitors can use the current SOTA model.
Lots of people can. Tools don't need to be top of the line to be useful. Snap-on may exist, but they don't put Harbor Freight out of business.
Advanced IDEs exist but complex projects were still built in vim.
The more capable the budget models get, the lower the marginal gains from using the frontier models, even if the frontier models always stay 6 months ahead.
DeepSeek models are open-source so there are a bunch of third-party providers offering similar prices. Factoring in that DeepSeek have to train the model (whereas third parties can make a small profit over just the inference costs) I'd assume that on net they're spending investor money, but I wouldn't think that's any less true of OpenAI.
Yes, DeepSeek is open-weight, but these third-party providers offering similar prices are subsidized with VC money as well. And you can find a range of prices for deepseek-v4-flash going up to and over $1/Mtok.
Even that $1/Mtok provided by Together AI is heavily subsidized by more than $1B in VC money.
This makes it unclear how the true cost curve is progressing. It’s not possible to confidently comment one way or another on the rate that cost is coming down when the entire industry is so heavily subsidized.
> Even that $1/Mtok provided by Together AI is heavily subsidized
Can you link this? I'm unable to find them offering deepseek-v4-flash. I think you could even host the pro model for a bit under $1/Mtok. You can get ~1000TPS out of the box on a B300 that you can rent for ~$3/hr, so around $0.83/Mtok.
Regardless - Alibaba, DeepSeek, NovitaAI, AtlasCloud, Cloudflare, DeepInfra, SiliconFlow, GMICloud, Morph, Baidu, Parasail, DigitalOcean, AkashML, StreamLake and likely others all seem to be offering it under $0.3 per million output tokens[0].
> This makes it unclear how the true cost curve is progressing
For no actual improvement in efficiency to be presented as a 10X yearly improvement since 2018, we'd need to currently be getting 100000000X more intensive models than we should be for what we're paying (a $1/Mtok model actually costs $100000000/Mtok). Presenting, say, a 9X actual yearly improvement as a 10X yearly improvement seems feasible, but for much beyond that I think the exponential just compounds too fast to reasonably fake.
The technology already exists now on the algorithmic front for the next 10x drop between everyone adopting DeepSeek's MLA, MoE (mostly already done), Medusa (a better version of Google's speculative decoding), Kimi's Attn Residuals, and Mimo's Sliding Window Attn, and (possibly) Microsoft's 1.58b (this may be a nothing burger).
Historically, algorithmic gains are only ~30% of the pie, but there's enough out there to get to 10x, with just what's available already. The other ~70% of the pie is better training data (often synthetic) and distilling frontier knowledge. There's no sign we are tapped out on that front.
MoE isn’t the magical improvement you think it is. Logprobs of MoE models are always worse in quality than the dense equivalent and they struggler harder at very long context quality than equivalent dense models. This is why Chinese companies like qwen are releasing dense and MoE versions of their models at near equivalent sizes. I always use/prefer the dense one.
Speculative decoding usually only improves decode and sometimes actually harm prefill and for agentic coding prefill matters more.
You’re right about the rest but I need to set the record straight on these details.
April of last year you'd get 1431 ELO[0] from o3-2025-04-16 for $8.00 per million output tokens. April of this year you can get 1436 ELO from deepseek-v4-flash for $0.2 per million output tokens.
[0]: https://huggingface.co/spaces/lmarena-ai/arena-leaderboard