[selectodude]

Some basic math supports it. A GB300 NVL72 is about $6.5 million. Lets say that you need $6 million worth of cooling and another $6 million worth of electricity. At current rates, that's 720 billion tokens worth of Claude Opus 4.7. At 100,000 tokens per second, it pays for itself in about 3 months.

Obviously this is an extremely rough calculation. I can even be off by a factor of 10 and it's still a pretty good return.

[overfeed]

Unless you're serving Chinese open-weight models - you have to consoder training costs. If you're off my 10x, then the amortization period is 30 months - far longer than the useful lifetimes of SoTA models. Frontier model development is a Red Queens race: you have to run as fast as you can, just to maintain your position.

[selectodude]

The discussion was if Anthropic makes money on inference. They do. They lose billions on training.

[joshuastuden]

No, because Anthropic can't serve their models unless they train them.

Training is akin to the cost of building the software/product. Inference is selling the product.

[BadBadJellyBean]

It's quite easy to sell something for a profit if you ignore the costs. Ultimate free money hack. I will start selling canned beans for the price of the beans plus a few cents. I will just ignore the cost of the cans, labor, power, machines, maintenance, distribution, storage and facility space. If I do that the few cents extra are pure profit.