[simonw]

Training costs are fixed. You spend $X-bn training a model and that single model then benefits all of your customers.

Inference costs grow with your users.

Provided you are making a profit on that inference you can eventually cover your training costs if you sign up enough paying customers.

If you LOSE money on inference every new customer makes your financial position worse.

[louiereederson]

Your ability to sign up enough customers is directly related to your ability to sustain training costs. The model runs have a short lifespan. They may serve many customers at a given point in time per run, but in order to serve those customers over time you need to continually spend on training.

I think your mental model for an LLM vendor is similar to a foundry (i.e. TSMC). They spend a bunch of R&D on developing leading edge nodes and build foundries. That in your mental model would be similar to training costs.

My point is the correct mental model is more like (but not exactly like) a SaaS company, ironically. SaaS unit economics are a function of gross margin, churn and acquisition costs, i.e. Revenue x gross margin / churn - CAC. My point is some element (maybe the entirety) of training costs are more like CAC than they are like TSMC's R&D and capex. The question to ask to test this view is: is what happens to OpenAI or Anthropic revenue in 2027 or 2028 if they stop spending on training today? My view is it'll drop precipitously. This implies churn is very high. It is true that training costs can be spread over customers though, so the analogy breaks down there, but I think it is a better mental model than the foundry one.

[simonw]

Yes, you need to continue spending money on training. But you don't need to spend MORE money on training just because you signed up more customers.