[surgical_fire]

It's impossible to tell at this point. I have no idea how much of their compute is also subsidized through deals they have with the hyperscalers, etc.

It's irrelevant how big their models may or may not be. Depreciation needs to be taken into account, so does actual compute expenses. Training those models is not cheap, and you will never reach a point where a model is "final". You will always need to train the next one.

Eventually the bill has to be paid. Money and resources are finite still.

[HDThoreaun]

Well the third party operators on open router are assuredly operating at a profit, including depreciation. The only reason they’d be profitable at 1/10 the price of the labs while the labs aren’t profitable is if inference costs the labs 10x as much per token

[surgical_fire]

> Well the third party operators on open router are assuredly operating at a profit, including depreciation

They don't train new models.

They have to depreciate their GPUs, which I hope they do.

[HDThoreaun]

I agree inference probably isn’t profitable yet if you include training costs. My claim is that marginal revenue from inference is higher than marginal cost. If that’s the case then if they scale enough the training costs will be amortized.

I realize I said assuredly when I meant assumedly. My mistake. I agree it’s possible that the third party open model hosters aren’t actually profitable, my claim does rest on the opposite.

[surgical_fire]

Mostly I am very skeptical on any numbers that claim these companies have a path to profitability. They raise way too much money way too often for the claim of profitability to make any sense. If inference profitable they could train new models at reasonable pace without raising tens of billions every few months. Neither they would be in a mad race for the IPO exit event.