[Tuna-Fish]

This is an example of common knowledge that is wrong. People look at their cash burn, assume that they spend this to subsidize inference, and get bonkers answers. Inference is not their largest expense.

Inference is cheap. Anthropic is only drastically subsidizing their plans if you count their training expenses as part of their costs.

[dathanb82]

If "inference is cheap," why is OpenAI spending a ton getting Broadcom to design custom AI chips that make inference cheaper? Reports suggest their custom silicon isn't all that good for training, it's all to make inference more efficient. That shouldn't be necessary if inference is already quite cheap.

[petra]

A large part of the market will be ad based. For that, having the lowest cost inference is useful.

Also for agent doing r&d, cheaper tokens allows doing more, which is always good.

[topranks]

But the training expense is part of their costs!

The question is can they just stop training at some point, fixing the models in time, and still have a useful product.

[therobots927]

Are you an anthropic insider or something? Because if you are you should delete this comment. If you aren’t then you don’t know what the hell you’re talking about.

[rcxdude]

For one point, you can look at the costs of similarly sized open-source models from inference providers (which are only making money on the markup on the compute), and compare with anthropic's prices. There's a pretty big price difference there and it would be hard to believe that anthropic's models are that much more expensive to run than those models.

[tomnipotent]

Those prices don't need to bake in the training cost, since it was eaten by someone else (whomever trained the open source models). Anthropic et al. need to price in the whole lifecycle.

[rcxdude]

Yes, but the point is specifically on whether inference in and of itself is profitable (i.e. whether the unit economics work out). They are still losing money overall, but they're not losing more money the more people use their product (quite the opposite: they need a lot of users at their fairly large per-token margins in order to justify their R&D spend, and the big question IMO is how strong the opposite side is: how much money do they need to spend on R&D to have a product that justifies such a high per-token premium?).

[xquce]

Surely the same can be said for the people saying the opposite?

[therobots927]

I didn’t make a claim. The parent explicitly said it was a misconception that inference is not profitable.

No one knows if it’s profitable or not so we’re left to speculate.

[Bnjoroge]

You can make a fairly decent assumption by calculating the margin on serving glm 5.2, and adding say 30% extra costs and it still leaves a healthy margin

[therobots927]

Where are you getting 30% from

[Bnjoroge]

It was a rough heuristic for how much more opus/5.5 would presumably cost if you extrapolate from glm5.2 prices. In any case input tokens for 5.4/4.6 are 70-100% more expensive, cached about 3-100%, and output tokens anywhere from 60%-240% more as per all their current api pricing. I highly doubt 5.4/4.6 are that much more expensive to serve given how cheap and commoditized inference has become, and how comparable they are perfomance wise.