[surgical_fire]

> For our language model benchmarking, we note that we consider endpoints to be serverless when customers only pay for their usage, not a fixed rate for access to a system. Typically this means that endpoints are priced on a per token basis, often with different prices for input and output tokens.

Okay, correct me if I am wrong, so this is measuring the inference costs for clients of AI services, not the the inference costs that the AI service itself has when they offer the service?

I mean, the other guy's claim is that inference costs had come down 20x-30x. But the analysis, if I understood correctly, is based on how much clients are paying for it, not how much it actually costs.

I can charge you 20x less for a service and have massive losses for it.

[simianwords]

It could be that OpenAI is subsidising their models by _fifty times_. Do you really think they are doing that? In some cases the costs went down by 200x. Do you really think OpenAI is subsidising their models by 200??

Its easier to just admit that technological advances helped decrease the cost instead of coming up with more complicated reasons like VC funding, subsidies and so on.

For instance take Deepseek and other opensource models - even they have reduced their costs by a huge margin. What explanation is there for opensource models?

[surgical_fire]

> It could be that OpenAI is subsidising their models by _fifty times_. Do you really think they are doing that?

Possibly. I don't know.

It could be unfeasible to increase prices so much whenever a new model was released.

Any assumption made here is based on vibes. I see no reason to drop my skepticism.

> Its easier to just admit that technological advances helped decrease the cost instead of coming up with more complicated reasons like VC funding, subsidies and so on.

They raised an absurd amount of cash, and still bleed money to an absurd degree.

VCs make money when they exit. OpenAI only needs to "make sense" until an IPO happens. Once private investors have their exit, the markets can be left to handle the resulting dumpster fire.

> For instance take Deepseek and other opensource models - even they have reduced their costs by a huge margin.

Chinese companies are very opaque. I don't pretend to have insight into it.

Is the company behind Deepseek profitable?

> What explanation is there for opensource models?

What opensource models have to do with inference?

Your argument is that training is expensive but inference is cheap (something I see no evidence of). Why would a company give away the expensive part of the work?

[simianwords]

>It could be unfeasible to increase prices so much whenever a new model was released.

This means you have no idea what I have been saying. A new model is costlier, but they release mini versions of old models that are way cheaper and compete with older models.

GPT 5 mini is way cheaper than GPT 4 but around the same performance

GPT-5 mini:

Input tokens: ~$0.25 per 1 M

Cached input: ~$0.025 per 1 M

Output tokens: ~$2 per 1 M

-----

GPT-4 (legacy flagship):

Input roughly $2.00 per 1 M

Output roughly $8.00 per 1 M

>Chinese companies are very opaque. I don't pretend to have insight into it.

False. The models are not opaque, you can literally download it and host it yourself. They have also released papers on how they reduced cost in certain areas.

This is literally them documenting the cost-profit ratio theoretical at 500%

https://github.com/deepseek-ai/open-infra-index/blob/main/20...

>The above statistics include all user requests from web, APP, and API. If all tokens were billed at DeepSeek-R1’s pricing (*), the total daily revenue would be $562,027, with a cost profit margin of 545%.

Not only that, there are other providers hosting these opensource models, there are so many companies - just go to openrouter.com

So this is your skepticism

- openai is subsidising their models so much that each year the keep doing it 20x and eventually reached 100x reduction

- all the investors are stupid and they still invest in openai despite unprofitability

- employees of openai and anthropic who have claimed that the unit costs are not high are also lying

- all other providers are in on the lie

- the chinese models like Deepseek is also in on the lie by posting research that is not plausible

- the fact that you can run models in your laptop today that beat previous years models is also not enough

[surgical_fire]

> openai is subsidising their models so much that each year the keep doing it 20x and eventually reached 100x reduction

If that's the truth, then originally they were subsidizing their models by the same factors.

This is not a great argument no matter how you cut it. And even then I would need to see evidence that this is true.

> all the investors are stupid and they still invest in openai despite unprofitability

Much to the opposite, those people are very smart. OpenAI can be extremely unprofitable and they can still profit massively through an exit event.

> employees of openai and anthropic who have claimed that the unit costs are not high are also lying

Possibly? Especially if they are in the position to profit in the case of an exit event, they would have every incentive to paint a rosier picture about the company.

> all other providers are in on the lie

I have no idea who you are talking about.

> the chinese models like Deepseek is also in on the lie by posting research that is not plausible

As I previously stated, I have no idea if Deepseek is profitable. By the looks of things, neither do you. Mentioning Deepseek's research is a non-sequitur.

> the fact that you can run models in your laptop today that beat previous years models is also not enough

This has no bearing on the cost of inference.