[vannevar]

Two issues with this. One, it's profitable assuming you just keep serving the same model forever, which is not realistic in this market. A given model has a shelf-life, which these days is measured in months, not years. Which means that trying to separate the cost of training the model from the cost of serving it doesn't make much business sense. And two, for providers that provide inference only via open weight models, the margins quickly move to commoditization. The "someday" when frontier model providers can enjoy their current high inference margins without the burden of significant training costs is never going to arrive.

[skybrian]

Commoditization means there's price competition. From a consumer perspective, that's good. You want it to be a low-margin, high volume, competitive business.

Although from a business perspective, it can end up being ruinous competition like solar panels or airlines. A stable equilibrium with prices neither too low or too high isn't guaranteed; it depends on market structure.

It's anyone's guess whether this reaches an equilibrium or not, but I still expect that there will be companies like OpenRouter and Fireworks that offer inference at reasonable prices.

[vannevar]

>Commoditization means there's price competition. From a consumer perspective, that's good. You want it to be a low-margin, high volume, competitive business.

You and I may want it to be a low-margin, high-volume business. But the valuations of OpenAI, Anthropic, and much of the rest of the AI industry are not based on that assumption. They are based on the assumption that there will be a couple of winners, like in the smartphone wars, and that those winners will be able to maintain good margins.

[skybrian]

There are low-end and premium smartphones, with different profit margins. Similarly, it seems like open weights models and high-end models could co-exist?

I don’t think there’s any way of knowing what the market structure will be in the end.

[vannevar]

>Similarly, it seems like open weights models and high-end models could co-exist?

Low-end and premium smartphones are still overwhelmingly either Android or Apple. And I'm sure open weights and premium models will co-exist. But that's not the future the current valuations are predicated on. I agree there's no way to say where the market structure will end, but I think we know enough to say where it won't end up. And some of the supporting players are starting to see the handwriting on the wall:

https://www.reuters.com/business/blackstones-qts-terminates-...

[credit_guy]

> A given model has a shelf-life, which these days is measured in months, not years.

Not all new models are trained from scratch. ChatGPT 5.3 to 5.4 (and likely 5.5) was basically the same model, but probably trained a bit more, not a new model from scratch.

> The "someday" when frontier model providers can enjoy their current high inference margins without the burden of significant training costs is never going to arrive.

That is debatable. I believe the moat for the frontier model providers is the compute. At the level of 10 trillion parameters (that Fable/Mythos are rumored to have), you need serious compute to serve inference, and you also need serious compute to train. Will DeepSeek, Qwen, Kimi, GLM come up with a 10T new model anytime soon? I doubt that. People keep saying that the Chinese labs are catching up to the US big 3, and measured in months the gap is now only 4-6 months. I doubt a Chinese version of Fable/Mythos will be released in the next 12 months.

[vannevar]

>ChatGPT 5.3 to 5.4 (and likely 5.5) was basically the same model, but probably trained a bit more, not a new model from scratch.

Then those models have an eroding moat and will be quickly driven down to commodity pricing. The only thing propping up inference margins are the cap-ex costs of training. That's the moat. That's why there's no way to win this game. You cannot have low training/infrastructure cost and high margins (such as would justify today's valuations).

>I doubt a Chinese version of Fable/Mythos will be released in the next 12 months.

I would take that bet.