[anthonypasq]

What is up with all this nonsense about token subsidies? Dario in his recent interview with Dwarkesh made it abundantly clear that they have substantial inference margins, and they use that to justify the financing for the next training run.

Chinese open source models are dirt cheap, you can buy $20 worth of kimi-k2.5 on opencode and spam it all week and barely make a dent.

Assuming we never got bigger models, but hardware keeps improving, we'll either be serviing current models for pennies, or at insane speeds, or both.

The only actual situation where tokens are being subsidized is free tiers on chat apps, which are largely irrelevant for any sort of useful economic activity.

[simonw]

There exist a large number of people who are absolutely convinced that LLM providers are all running inference at a loss in order to capture the market and will drive the prices up sky high as soon as everyone is hooked.

I think this is often a mental excuse for continuing to avoid engaging with this tech, in the hope that it will all go away.

[kingstnap]

I agree with you, but also the APIs are proper expensive to be fair.

What people probably get messed up on as being the loss leader is likely generous usage limits on flat rate subscriptions.

For example GitHub Copilot Pro+ comes with 1500 premium requests a month. That's quite a lot and it's only $39.00. (Requests ~ Prompts).

For some time they were offering Opus 4.6 Fast at 9x billing (now raised to 30x).

That was upto 167 requests of around ~128k context for just $39. That ridiculous model costs $30/$150 Mtok so you can easily imagine the economics on this.

[louiereederson]

Referring to my earlier comment, you need to have a model for how to account for training costs. If Anthropic stops training models now, what happens to their revenues and margins in 12 months?

There's a difference between running inference and running a frontier model company.

[simonw]

Training costs are fixed. You spend $X-bn training a model and that single model then benefits all of your customers.

Inference costs grow with your users.

Provided you are making a profit on that inference you can eventually cover your training costs if you sign up enough paying customers.

If you LOSE money on inference every new customer makes your financial position worse.

[louiereederson]

Your ability to sign up enough customers is directly related to your ability to sustain training costs. The model runs have a short lifespan. They may serve many customers at a given point in time per run, but in order to serve those customers over time you need to continually spend on training.

I think your mental model for an LLM vendor is similar to a foundry (i.e. TSMC). They spend a bunch of R&D on developing leading edge nodes and build foundries. That in your mental model would be similar to training costs.

My point is the correct mental model is more like (but not exactly like) a SaaS company, ironically. SaaS unit economics are a function of gross margin, churn and acquisition costs, i.e. Revenue x gross margin / churn - CAC. My point is some element (maybe the entirety) of training costs are more like CAC than they are like TSMC's R&D and capex. The question to ask to test this view is: is what happens to OpenAI or Anthropic revenue in 2027 or 2028 if they stop spending on training today? My view is it'll drop precipitously. This implies churn is very high. It is true that training costs can be spread over customers though, so the analogy breaks down there, but I think it is a better mental model than the foundry one.

[simonw]

Yes, you need to continue spending money on training. But you don't need to spend MORE money on training just because you signed up more customers.

[louiereederson]

Anthropic reduced their gross margin forecast per external reporting (below) to 40%, and have exceeded internal forecasts on inference costs. This does not take into account amortized training costs which are substantial (well over 50% of revenue) and accounted for as occurring below gross profit. If you view training as a cost of staying in the game, then it is justifiable to view it as at least a partially variable cost that should be accounted for in gross margin, particularly given that the models stay on leading edge for only a few months. If that's the case then gross margins are probably minimal, maybe or negative.

https://www.theinformation.com/articles/anthropic-lowers-pro...

[nerevarthelame]

> Dario in his recent interview with Dwarkesh made it abundantly clear that they have substantial inference margins, and they use that to justify the financing for the next training run.

You're putting way too much faith in Dario's statements. It wasn't "abundantly clear" to me. In that interview, prior to explaining how inference profits work, he said, "These are stylized facts. These numbers are not exact. I'm just trying to make a toy model," followed shortly by "[this toy model's economics] are where we're projecting forward in a year or two."

[anthonypasq]

so you think Dario was just straight up lying that each model re-coups its training costs and is profitable? In order for that to be the case, inference just has to have good margins. If you just do some basic math and compare it to chinese open source models, theres just no way Sonnet is actually as expensive as API costs indiciate.