[surgical_fire]

I have no idea if costs indeed came down 20x-30x.

This claim sounds extremely fancy when AI companies bleed money, and will keep bleeding money in the foreseeable future.

I don't pretend to know the future. Maybe LLMs become economically viable and are the future, maybe not. I don't really care either way, to be frank.

And I use LLMs, btw. I pay for a ChatGPT account, but I find it only moderately useful. I always sort of question myself upon renewal date if it is worth the 20 bucks I spend monthly on it.

In no small part I keep using it to keep myself up to date on the best practices of using them in case it becomes standard.

[simianwords]

https://epoch.ai/data-insights/llm-inference-price-trends

Do you have any reason to not believe it? It’s expected for costs to come down

[surgical_fire]

The graph you linked seems to compare different OpenAI models in terms of "price per million tokens".

I am very skeptical of any financial information that comes from OpenAI. I have no idea how truthful those numbers are, or how creatively they can be collected to paint a rosier future for them.

Even if the numbers are truthful, I have no idea how the calculate price there. Is it in terms of cost of compute they rent? Is this cost subsidized or not?

Also, I don't know this "epoch.ai" website, I don't know their stance. The website name itself does not inspire my confidence on their reporting of anything related to AI. "Eat meat, says the butcher" vibes and all.

You can claim that the AI bleeds money because training is expensive, but inference is cheap. So it will only be financially viable when they stop training models? So they would need to stop improving their capabilities entirely for it to make any sense, is that your claim?

Even if I take this claim at face value (and that would take a lot of faith I don't have to give), it doesn't sound as good as you think it does.

[simianwords]

>To analyze the decline in LLM prices over time, we focused on the most cost-effective LLMs above a certain performance threshold at each point in time. To identify these models, we iterated through models sorted by release date. In each iteration, we added a model to the set of cheapest models if it had a lower price than all previous models that scored at or above the threshold.

Can you look at the analysis? It will make it clear. I mean its so obvious because GPT 4 costs way more than GPT 5.2-mini but much worse performance.

>Even if the numbers are truthful, I have no idea how the calculate price there. Is it in terms of cost of compute they rent? Is this cost subsidized or not?

Do you think they are subsidising 900x or simply that the costs have gone down?

Overall you have shown what I feel is extreme skepticism in something that is obvious. You can literally run a model in your laptop that matches an older closed model. Costs are obviously going down, I have shown data. Use your own anecdotes and report.

Extreme skepticism in such a way doesn't do any help.

[surgical_fire]

> Overall you have shown what I feel is extreme skepticism in something that is obvious.

I think you show extreme faith in something that is very obscure.

For me to believe in the analysis I would need to trust the numbers that the analysis is based upon. I see no reason why I should trust this. What sort of regulatory body or neutral third party inspects those numbers to ensure they are not a fabrication?

But you can claim I am a hater if it justifies your worldview. Skepticism is sinful for the believer.

[Kye]

>> "The dataset for this insight combines data on large language model (LLM) API prices and benchmark scores from Artificial Analysis and Epoch AI."

I don't know about Epoch AI, but Artificial Analysis shares its methodology: https://artificialanalysis.ai/methodology

Their chart of inference prices split by benchmark intelligence: https://artificialanalysis.ai/trends#efficiency

[surgical_fire]

> For our language model benchmarking, we note that we consider endpoints to be serverless when customers only pay for their usage, not a fixed rate for access to a system. Typically this means that endpoints are priced on a per token basis, often with different prices for input and output tokens.

Okay, correct me if I am wrong, so this is measuring the inference costs for clients of AI services, not the the inference costs that the AI service itself has when they offer the service?

I mean, the other guy's claim is that inference costs had come down 20x-30x. But the analysis, if I understood correctly, is based on how much clients are paying for it, not how much it actually costs.

I can charge you 20x less for a service and have massive losses for it.

[simianwords]

It could be that OpenAI is subsidising their models by _fifty times_. Do you really think they are doing that? In some cases the costs went down by 200x. Do you really think OpenAI is subsidising their models by 200??

Its easier to just admit that technological advances helped decrease the cost instead of coming up with more complicated reasons like VC funding, subsidies and so on.

For instance take Deepseek and other opensource models - even they have reduced their costs by a huge margin. What explanation is there for opensource models?

[surgical_fire]

> It could be that OpenAI is subsidising their models by _fifty times_. Do you really think they are doing that?

Possibly. I don't know.

It could be unfeasible to increase prices so much whenever a new model was released.

Any assumption made here is based on vibes. I see no reason to drop my skepticism.

> Its easier to just admit that technological advances helped decrease the cost instead of coming up with more complicated reasons like VC funding, subsidies and so on.

They raised an absurd amount of cash, and still bleed money to an absurd degree.

VCs make money when they exit. OpenAI only needs to "make sense" until an IPO happens. Once private investors have their exit, the markets can be left to handle the resulting dumpster fire.

> For instance take Deepseek and other opensource models - even they have reduced their costs by a huge margin.

Chinese companies are very opaque. I don't pretend to have insight into it.

Is the company behind Deepseek profitable?

> What explanation is there for opensource models?

What opensource models have to do with inference?

Your argument is that training is expensive but inference is cheap (something I see no evidence of). Why would a company give away the expensive part of the work?