[mediaman]

The (quite excellent) article discusses several of your points. If you haven't read it, I recommend it.

  - Commodity market profitability is determined by marginal cost of production. LLMs have marginal cost; traditional software does not.

  - Models are not free. Downloading them is free. Running them is not. This has manufacturing economics, not software economics; the idea that they are "free" is an economic category error as it relates to their actual use

  - The highest tier Chinese models are not more economical than US frontier models. Try GLM 5.2 and see how much it costs to do real work. I did, and it was more expensive than GPT 5.6.

  - This is because US labs are leading on cost efficacy of inference ($/task)

  - Training will decline as a percentage of costs as inference expands compute share due to agentic workloads. A big part of training now is optimizing token efficiency. It's hard to distill token efficiency; that is perhaps why Chinese LLMs are so inefficient.

  - With increasing inference as % of total compute, if labs create efficient models -- which they can, because they can create highly optimized models amortized over very high inference loads -- they can be low cost producers, and be competitive at $/task rates

OpenAI really shows the way here. Their cost per task is less than half that of Anthropic because of more efficient tokenization and less verbosity. OpenAI is both cheaper and better than Chinese models for frontier work.

[larrysalibra]

Ben's article "distills" down to 2 reasons that US frontier labs shouldn't be "afraid":

1. US frontier lab unit economics are better 2. US frontier labs are moving up the stack making tools that are "stickiness" and will prevent users from switching.

For 1...he doesn't provide any evidence for US lab unit economics being better...the major input to unit economics is electricity...which is cheaper in China. And building data centers and connecting them to electricity is both cheaper and an order of magnitude faster in China. The main input that US labs might have an advantage in is in cost/access to chips, but that given the level of chip investment in China it seems unlikely to hold.

For 2...there's little evidence these tools are sticky. At least in programming, the trend seems to be tools like opencode that support multiple models and providers.

And even when they are sort of sticky, as we know on hacker news, people figure out how to point the tools they like to competing models even when the app doesn't official support it.

And every improvement in model capability makes it increasingly easier to make your own tools.

Wrote more on this in a blog post that has an earlier HN discussion: https://news.ycombinator.com/item?id=48982061

Direct link: https://larrysalibra.com/ben-thompson-is-wrong-us-frontier-l...

[gmerc]

He’s glossing over the reason they are not: 90% profit margin of Nvidia. Power is only a small part, single digit, it will eventually matter but does not really today.

What is the cost of AI? The single largest ingredient is Nvidia profit margin.

Huawei accelerators are not as efficiency yet, but they don’t nearly extract as much margin.

Why would future revenue stay with the labs given this situation? This whole thing had an airline industry sized red flag on it that makes investing into frontier lab about as sexy as investing in United.

Maybe the token economy is some kind of reverberation of the airline reward miles economy, the emergency hatch to be able to survive under maximal supplier extraction (Nvidia is just the top of a monopoly stack here, even if they replace those chips, the HBM, ASML, Foundry layer can get their dues)

[michaelt]

> Power is only a small part, single digit, it will eventually matter but does not really today.

Sorta yes, sorta no.

A single 5090 consumes 450W - at Californian energy prices of $0.38 per kWh that's $0.17 per hour. And the card itself costs $4100 on amazon. So after 2.75 years running at full power 24/7 you'll have spent more on electricity than on the card. I would have thought most data centres being built today would have a design life longer than 3 years.

Of course you can throttle the cards to ~300W without losing too much performance. But also you need more than a single 24GB card to run most modern LLMs.

[verall]

DC has wider margins than the 5090. by someone elses rough numbers (https://www.spheron.network/blog/gb300-nvl72-vs-gb200-nvl72-...) for GB300 rack:

> 132kW

> ~$3.7-4M

So about 300x the 5090's power but 1000x the price. Roughly 9 years for electricity to exceed price at $0.38 and datacenters will show up in areas with cheaper power than CA.

[Omniusaspirer]

Anyone seriously building out AI infrastructure I presume is paying nowhere near $.38/kWh which is extortionate. Utility scale solar is closer to $.02-.03/kWh, then maybe around ~$.10/kWh for natural gas peaker plants.

[londons_explore]

Utility scale power price varies widely by location and exact time of day.

LLM's aren't very latency sensitive and can therefore move to wherever power is cheapest.

Right now that's places next to aluminium smelters (which also like very cheap electricity 90+% of the time).

[inigyou]

GPU design life is commonly cited as about 3 years. The main driver is not physical degradation of the cards but the expectation they'll become obsolete with newer cards.

[threatripper]

Also add electricity 50% on top for cooling the DC.

[aurareturn]

Cost of electricity isn’t a long term advantage in my opinion. Private companies will figure it out.

What matters most is $/completed task. It does seem like OpenAI and Anthropic are winning here even with worse electricity rates. Perhaps it is made up by the efficiency of Nvidia and Broadcom chips, which China can’t get in mass.

I do think that OpenAI and Anthropic are moving up in stickiness. My company has rallied around Claude. We are customizing Claude Code, adding knowledge bases for non technical people, writing skills for them, using Claude features company wide. It’s hard to move.

Meanwhile, I personally use ChatGPT outside of work. The memory, ease of use, habit keeps my subscribed.

[blensor]

I'm a model nomad, using whatever solved my last problem the best and where it makes the most sense to start my next work in.

However with the latest models Fable, Kimi K3, 5.6, it's getting to a point where I sometimes forget what model I am on without noticing a difference. And once I realize it because something may not be exactly like I expected it I won't switch for that work either because I don't want to invalidate the cache.

For the next work I will do there is maybe a 50/50 chance to remember to switch the model before I start.

That's not what I would call stickiness towards a certain provider.

[amelius]

You didn't say what kind of problems you solve with AI. It matters a lot if you are doing HTML versus C++, for example.

[blensor]

In no order of importance:

  - Refactoring a 13 year old in-house vacation rental booking system ( python/turbogears )
  - Backend development for our VR fitness game ( flask/python )
  - Unity development on our VR fitness game ( C#/Unity )
  - VR game development experiments ( Godot/GDScript )
  - Standalone SLAM localization service ( C++ )
  - Audio analysis ( python/pytorch )
  - Virtual display with Viture display glasses ( C )
  - Reverse engineering a library I am using for another project ( ghidra -> C - no MCP yet, that's something I am looking forward to )
  - Public facing website rebuilding for the booking system above ( PHP/JS )
  - Generative 3D environments for our VR fitness game ( python )
  - Wireless camera/IMU based tracker for the SLAM system ( C )

Once I've dug in with a specific model into a problem I tend to stick to that because I have a feeling what it will do and how well it works, but when I start a new thing I usually use whatever the model was last set to.

[amelius]

Wow, now we're talking :)

[wbadart]

Seems like most popular harnesses, including codex and Claude code, support Agent Skills (an open spec for skill formatting/ organization): https://agentskills.io/clients

Which is to say, this isn't really a lock-in/ stickiness vector (unless maybe the wording itself of a skill is hyper-optimized for a specific model)

[littlecranky67]

you can simply switch to z.ai/GLM-5.2 inside Claude Code by settings env variables in .claude/settings.json

[culi]

> Private companies will figure it out.

Across sectors, China added 543 GW of energy in 2025. Next year, USA is expected to add between 70 and 80 GW of energy

[metalspot]

You really have to look at energy/capita and how much energy is embedded in exports. The gross numbers are misleading. The US wasn't building new electrical generation capacity because it didn't need it and there was no market for it (caveats apply, but in a broad sense this is the major reason). Now that the market exists the question is how much can the US actually bring online and how rapidly, which is a real challenge after decades of degrowth politics used to justify slash and burn consumption of the industrial base.

AI is really all about electricity. AI could be completely fake and yield zero value whatsoever and the US would do exactly what it is doing now because the AI bubble is what creates the market for building new electrical generation capacity, which is needed for re-industrialization. Also why our friends in UK/Europe/China are so busy pushing anti-AI propaganda to try to undermine this.

[culi]

> The US wasn't building new electrical generation capacity because it didn't need it

You need to understand that the US added 40.3 GW of energy capacity in 2023. 70 to 80 represents a dramatic growth (nearly doubling) in new capacity.

Also China is just getting started. 8 out of the 12 nuclear power stations opening worldwide in 2027 are Chinese https://world-nuclear.org/information-library/current-and-fu...

If AI is "all about electricity" as you say then the US was barely ever in the running in the first place. There's no possible way the US could compete any time in the next 2 decades (putting aside a dramatic shakeup like war)

> Also why our friends in UK/Europe/China are so busy pushing anti-AI propaganda to try to undermine this.

Lol people make up the funniest theories when a political idea they don't like is gaining popularity. At least you didn't blame Russian bots

[21asdffdsa12]

Oh, the UK and Europe castrated themselves and want others to follow the example. They still don't have made the connection between "I give up ability" and i get attacked by a proxxy opponent by those i gave ability up too.

They do not want to life in the world that is and thats going to be, but in the past and the world they green ideology promised. Reality denial be a addictive poison.

[flir]

That last para is a novel idea. I wonder if there's any evidence to support/undermine it though?

[metalspot]

The concepts of industrial reserve capacity and using dual-use consumer goods to subsidize military production capacity are well known and widely practiced historically in the US. China adopted this strategy from the US, and the US conveniently forgot about it for a few decades in order to justify selling off the industrial base to China, but at least based on public documents like the published U.S. National Security Strategy, I would assess with high probability that this is explicitly recognized and being followed now.

AI is not fake and it does work, but what I am saying is that from a pure systemic analysis perspective, you can do the numbers, and even if AI was complete fugazi, the benefits you get from the electrical generation capacity, and the ability to fund it through private markets, which bypasses Congress, and locks in commercial contracts (often with foreign governments) which will be almost impossible politically to reverse, would still make it optimal from a strategic perspective. That is my calculation, and to the extent that it is correct, I would assume that the US Military's strategic planning apparatus would arrive at the same conclusion.

AI compute has some unique characteristics that make it especially useful for grid management. Moving consumer compute to the cloud means that the electrical use of that compute can be centrally managed. In an emergency, you can cut electrical use for consumer AI by 50% or more, because chips run more efficiently at lower power, and you can shift workloads onto quantized models, reduce resolution for video output, etc, to reduce compute, which leads to minor service degradation but not interruption. AI datacenters are also adding massive amounts of battery storage capacity, which is an additional grid buffer. For every GW in capacity added by hyperscalers that is creating a dispatchable reserve capacity of 50% under completely normal circumstances (hyperscalers do this internally to optimize their own costs) and then that number goes up depending on the scale and duration of the emergency.

[anonzzzies]

They are not stopping either.

[golem14]

I'd really love to see the evidence on this!

[HarHarVeryFunny]

> 1. US frontier lab unit economics are better

That's not generally true, since there is generally still much reliance on NVIDIA. The true low cost providers are Google with their TPU and vertically optimized stack, and Amazon with Trainium. However, Google does not have their own frontier model, and Anthropic (who are partially served by Amazon) are also paying a premium for extra NVIDIA-based capacity from SpaceX, maybe soon from Meta too.

I don't know how the economics of domestic Chinese Huawei-based clouds (no NVIDIA) compares to the west, but since serving cost is mostly hardware depreciation and to a lesser extent electricity, they are not necessarily at a disadvantage (Ascend 950 costs roughly 50% of an NVIDIA H100), and more to the point it is irrelevant when considering US commercial use that is more likely to be using Chinese open weights models from US providers served on NVIDIA based hardware.

I think the real significance of Chinese frontier models being open weight is that it takes development cost amortization out of the US-based serving cost, while the US AI labs can't afford to do this. The US labs therefore need to reduce development spending to remain price competitive. The Chinese companies are of course still making money from the Chinese market, whether by selling API access or by other business models such as Ziphu making 75% of it's total revenue by selling services to Chinese customers who are running their models on-prem due to the Chinese apparently being very concerned about data privacy.

[pishpash]

More basically, production cost matters only if inference is priced at commodity prices. That's not what VC's signed up for, which is rent-seeking.

[Computer0]

In a corporate setting yes Opencode all the way. However in a non corporate setting I am getting $3000 of api usage a month for $100 at Anthropic and only use open code for the smallest cheapest tasks

[tvink]

I think calling opencode the trend is naive. This not what is being run on company time.

[hack1312]

OpenCode is absolutely used on company time.

[sciencejerk]

Shhhhhh...! OpenCode only runs on authorized machines by responsible employees following company policy ;)

[striking]

Sure, let's have a look...

> I highly doubt that Chinese models are cheaper to serve on a marginal cost basis, they just seem cheaper because Anthropic and OpenAI are so supply constrained that they are charging far more than they would if there were sufficient supply to meet the demand for intelligence. [emphasis mine]

I guess I'm missing the part of this article where they bring hard numbers in to back up the argument here. What work was attempted? https://cursor.com/evals shows the previous generation of open models (Kimi K2.7) trading blows with the others, cost effectively. Composer 2.5 is itself a fine-tune of K2.7, and it's apparently quite token efficient, so why would it be impossible for a Chinese lab to achieve something similar? GLM 5.2 Max is also ranked above the lower end OpenAI models and is not far off in price.

It's weird to have this entire discussion about tokenomics without mention of the circular financing and debt raised by labs in the West, which can then essentially give away their capacity to end users. OpenAI giving away quota resets to subscribers like candy on Halloween while their compute partner Oracle's bonds is reevaluated to be one grade above junk? How?

I don't think you can make an argument about the future one way or another by arguing using the listed prices. The math is not internally consistent enough for it.

[gruez]

>What work was attempted? https://cursor.com/evals shows the previous generation of open models (Kimi K2.7) trading blows with the others, cost effectively

Because you're comparing retail price whereas the parent commenter (and the article) is talking about marginal (ie. inference) costs. American labs are providing a premium product and they're charging accordingly. Meanwhile for chinese models they're open weight so they're limited to how much they can charge without competitors undercutting them.

If we use tokens as a rough proxy of inference costs (rough approximation, I know) and look at artifical analysis benchmarks, you see that all the open models are behind the pareto frontier in terms of efficiency.

[striking]

I'm arguing we can't trust retail prices because the marginal pricing isn't meaningfully connected to it anyway.

But if we have to look at what we think margins might look like, DeepSeek continues to host v4 Flash at the existing price despite competitors beating it in price (https://openrouter.ai/deepseek/deepseek-v4-flash), so there's at least one example of a Chinese lab charging a predetermined price despite competition. And no one but Moonshot is hosting Kimi K3 yet (https://openrouter.ai/moonshotai/kimi-k3). Perhaps there's room in the market for those who release their models to make margin on them.

And I believe my Composer example speaks for itself. The open models are behind but there's tangible proof they can be tuned for pareto frontier efficiency. See "Cost per Task" at https://artificialanalysis.ai/agents/coding-agents.

[gruez]

>DeepSeek continues to host v4 Flash at the existing price despite competitors beating it in price (https://openrouter.ai/deepseek/deepseek-v4-flash),

their competitors are discounted at around 33%, so it's safe to say that's the margin, maybe less if their competitors have worse caching or quantization. Meanwhile claude code/codex resellers selling tokens for 90% off API price, presumably by reselling usage from fixed consumption plans, which gives an idea on how fat the american labs' margins are.

>And I believe my Composer example speaks for itself. The open models are behind but there's tangible proof they can be tuned for pareto frontier efficiency. See "Cost per Task" at https://artificialanalysis.ai/agents/coding-agents.

But composer is a closed model? If it's really that easy to get better coding performance, why haven't the chinese labs replicated it? And this is all assuming the performance boost is real and not from benchmaxxing. Moreover if you apply the "street price" discount I mentioned above, American labs look far more favorable.

[striking]

The fixed consumption plans are offering several times their worth compared to API pricing with completely free cache reads: https://she-llac.com/claude-limits

I look at that and think that they must be losing money hand over fist on something like this, not that this shows what their margins are like. If their margins are like this then I don't see why they'd be raising money and shuffling it around in circles.

> If it's really that easy to get better coding performance, why haven't the chinese labs replicated it?

Nobody said it would be easy! I just think it's possible, and that presumably they will get around to doing it at some point.

[ycui7]

discounted competitor could cheat. they can offer subpar model response and sell it as deepseek-v4. it is uneconomical to prove inference providers are cheating, so they get away with it. cheating inference provider does not care if their customers stay.

[c0brac0bra]

The 33% discounted competitors have no non-retention policy

[byzantinegene]

the reselling of fixed price plans by resellers are causing american labs to lose alot of money and that's why they're trying really hard to stamp it down.

[metalspot]

Deepseek is still charging cached input at 1/10th the price of any competitor.

For an example of my real token usage for a day with DS: Input (Cache hit) 530,949,760, Input (Cache miss) 7,875,004, Output 1,389,685 - it is still 1/5th the price of Baidu (the cheapest) and 1/7th-1/10th the price of US hosts.

Also, Deepseek platform is not the same thing as Deepseek open weights. There is a major misconception that the existence of an open weights model means that it is the same thing as the proprietary platform offering, but that is definitely not the case.

[ehnto]

US running costs are higher than in China, because the US lags behind in energy, has higher real estate costs, and wage costs are higher.

Eventually we will hit a "good enough for cheap enough" and frontier models will hit diminishing returns (if they haven't already for a lot of types of work)

Don't think the rest of the world will sit on their hands while the US soaks up chips either, demand gets filled and if the US won't fill global demand for chips that's an opportunity to undercut again.

The other thing the rest of the world doesn't have to fund is the ridiculous valuations on these companies.

Unless you think the US can stay ahead just with model efficiencies, and that no one else will eventually match them, you are looking at the writing on the wall.

All that to say, the rest of the world is more than willing to eat your lunch, they have a dozen good reasons to, and they're already showing good results.

Just on the economics side, we've been here before too, US companies typically export their commoditization and live on brand royalties. Think all the cheap manufactured goods, the US doesn't make any of it. That's because the US can't compete on margins for numerous reasons, it's too expensive, I don't think AI is any different here except that the brands are currently valued in the trillions and I suspect that greed will be their undoing.

[analyte123]

The US does not lag behind in energy. Industrial electricity prices in most places in the US are competitive with China, or even cheaper.

[ehnto]

Apologies, I meant ability to deploy new generation. You're right that areas of the US are cost competitive.

This can change quickly though, so it's not that big of a deal. If AI energy demands push the Gov to deregulate/fast track new plants, or the industry decides to build out their own generation renewables.

[zx8080]

Links please.

[analyte123]

IEA’s chart for “final electricity price for large industrial customers in energy-intensive industries” [1] shows the US cheaper than China, but this is using data from Texas as representative. I’m not sure how they back this representation up, or how different this is from GPP “business” prices. So “most places in the US cheaper than China” may be wrong, but certain states or regions are at least competitive.

[1] https://www.iea.org/reports/electricity-2026/prices

[mapontosevenths]

This is very misleading and almost certainly deliberately so.

Texas is the only is State that doesn't participate in the national grid (90% anyhow). Their prices are lower because they dont abide by the regulations that they would be required to if they did. Like weather proofing infrastructure.

This is also why their grid crumbles every time it gets cold, or hot, or... Tuesday.

See the 2021 grid failure for a particularly bad example. In that instance somewhere between 250-700 Texans died to keep their prices low.[0]

[0] https://en.wikipedia.org/wiki/2021_Texas_power_crisis

[mapontosevenths]

This seems to show that even with China taxing industrial electricity to subsidize household electric bills it's still 30% cheaper for industrial electricity in China. [0] Those same taxes make household electricity about half the price of US power.

If I had to wager why, I'd say it's due to embracing solar on massive scales recently. Only a few years ago the US was competitively priced.

[0] https://www.globalpetrolprices.com/compare_countries/USA/Chi...

[actionfromafar]

But the clean, beautiful, coal will come online soon right?

[appplication]

I think there are some really interesting thought there, but I’d challenge some of this:

> Models are not free. Downloading them is free. Running them is not. This has manufacturing economics, not software economics; the idea that they are "free" is an economic category error as it relates to their actual use

I think a large part of manufacturing economics is illiquid overhead and the cost of expertise to set up and run your manufacturing line. Compute economics don’t have the same illiquidity nor do they require the same expertise or even specialized infra (current temporary chip shortage aside).

The implications of this are small players (e.g. your uncle running an inference server out of his garage) have comparably efficient marginal costs as big players. Compare this to actual manufacturing where small players have essentially no access to the manufacturing facilities of the big players.

Additionally, big players with a lot of compute who are not meaningfully in inference today (e.g. Amazon) have a fairly straightforward glide path to utilizing that compute to compete.

> This is because US labs are leading on cost efficacy of inference ($/task)

It’s possible, but I would need to see better data on this.

>A big part of training now is optimizing token efficiency. It's hard to distill token efficiency; that is perhaps why Chinese LLMs are so inefficient.

I think it’s fair to assume this is true, but also token efficiency is not a meaningful competitive moat. It’s not like these are secrets the Chinese will never figure out, it’s a fairly active research space and the outcomes are quantifiable.

[senderista]

Amazon is not "meaningfully in inference"? Bedrock seems to have a ton of enterprise customers, some of which would never trust the AI labs themselves with their data but will trust Amazon.

[appplication]

Relative to their other compute or other players inference, not as significantly. Though yes, they certainly have some share.

[cdud3]

A lot of this companies are also in parallel evaluating running Chinese models inhouse to be independent of the good will and profit margins of externals.

This is where Oracles "datacenters for rent to run your own Chinese models" strategy will benefit. The LLM SaaS game is a lost one thanks to China.

[lemax]

But this assumes Chinese models will not achieve token cost optimization. Intelligence needs are fairly flat for many tasks, and the Chinese models have caught up on this front. Next they achieve greater token cost efficiency and we don’t need OpenAI.

[overfeed]

That the author doesn't acknowledge the relentless R&D efforts DeepSeek has been plowing into optimization, and giving a default win to OpenAI/Anthropic on the supposition that they've been serving models for longer is a black mark against the article.

I appreciate the transparency in explicitly stating their motivation for writing the article (a response to what the author saw as an overreaction to Chinese models), but I feel the article goes too far the other direction, with multiple unsupported leaps of logic, and overstating the stickiness of AI client products.

[VulgarExigency]

The model that is most optimized around token cost is, in fact, Chinese. DeepSeek is astoundingly cheap by default, but if you use it from Reasonix (the harness optimized around its cache), it becomes even cheaper.

[ryeguy]

I keep seeing mention of the cache, what's special about it? All frontier llms have prefix caching, what is special about deepseek's approach?

[fspeech]

Their kv cache is smaller so they can use less vram and also keep your prefix cached for longer. https://deepseek.ai/blog/deepseek-v4-compressed-attention

[metalspot]

This also comes with significant capability reduction. deepseek-v4-flash is very good in the < 250K range, then degrades between 250-500K, and is practically unusable after 500K.

[edit]

This is my observation from using it without an specific context engineering to optimize for Deepseek's cache compression and sparse attention mechanisms. I am pretty sure that if you specifically structure your context to align to the cache compression boundaries you can significantly improve performance in the full 1M context, but there is not much reason to do this, because if you design your outer loop to work with shorter contexts that solution is portable and more efficient, so I haven't bothered with a optimizing for DS at this point.

[Der_Einzige]

Btw - assuming NeurIPS reviews aren’t garbage tomorrow, I’ll have a paper out which claims that most long context problems in models are really sampling problems in disguise

Switch to a modern sampler like min_p or ideally a better one like top-n-sigma (it’s in llamacpp) and your “my model gets stupid at long context problems” will basically go away.

Unfortunately this fact is still not well appreciated yet despite nearly every modern sampling technique getting an oral wherever they get presented. Min-K just got an oral at ACL 2026, for a hyper recent example of this. There’s a reason they keep getting orals.

The field massively ignored sampling for mostly safety reasons and now the whole field incorrectly believes long context doesn’t work on small models. Long context is an out-of-distribution problem. Your sampler configured properly keeps you in distribution.

Oh and this is doubly true for quantized models. I run my qwen 3.6 27b with 4bit quants from unsloth and get excellent performance because my sampler stack is good and not the garbage that is top_p and top_k. Also, yes, you need to ignore the trash recommended sampler settings from the Chinese labs (they’re wrong/bad).

[SoftTalker]

> Models are not free. Downloading them is free. Running them is not.

Is this really different from traditional software? Downloading postgres is free. Running it is not. You either buy hardware and assume the costs of owning and running that, or you pay to run it in the cloud.

[aurareturn]

I think the point here is that it takes the same hardware to inference an open source model as OpenAI/Anthropic inference their models.

IE, a lower param OpenAI/Anthropic model can compete with a higher param open source model.

So even if you are an American company who downloaded Chinese models in hopes of saving in cost, you still have to beat OpenAI and Anthropic in $/task which is very tough to do over the long run.

[Gigachad]

The problem is these AI companies are running at a loss with the hopes of pumping the prices after everyone is hooked. Now they are locked in at selling API access at rock bottom price. The valuations won't hold.

[elictronic]

It’s even worse. There operating costs are rising because of their own demand claims while those same rock bottom api prices are locked in.

There being squeezed by their own stock pumping and SpaceX pretending to be an AI company is driving down the exit strategy. I’m guessing one starts going full Theranos and begins claiming full AGI or gets the US government to government cheese then hard. It’s going to be a few crazy months.

[pishpash]

No they don't. Are OpenAI and Anthropic charging at cost, or wish to? No.

[aurareturn]

If you have more efficient models, you can reduce price and grab market share or keep price and increase your margins.

Over time, this compounds. More profits means more investments. More market share means more control.

[charlieflowers]

But inference costs scale per task whereas platform services like postgres typically amortize across tasks. If you are selling tasks done by inference, then compute is part of your COGS and it scales per task.

[SoftTalker]

Much more so, I would agree. A bit like how mainframe batch jobs might have once been charged back based on "cycles" used for a job or other tracked metrics.

[ChaitanyaSai]

The thing I do not understand here because it seems obvious: AI will be a commodity market and you simply cannot have a large PE multiple. So the valuations imagine a global commodity monopoly or duopoly coupled with the increased intelligence still disallowing other suppliers from becoming competitive? Without any network effects to help?

[horacemorace]

Perhaps to moneymen the difference between “ChatGPT” and the technology behind it isnt’t obvious. I’ve been very surprised at how few otherwise smart people are completely in the dark about how capable current models are.

As soon as manufacturing starts building this stuff more, it will commoditize. The hardware prices won’t be terribly larger than the original. We’ll have a “Bambu labs” style company to make the AI OS, whatever that is.

[jppope]

There are many ways to create "sticky" products, even commodity products (e.g. coca cola, starbucks, etc). Network effects are just one.

Regarding Price to Earnings... I'm not sure everyone fully fleshed out the end game for the frontier companies. It appears the pitch is that this current phase is a stepping stone to Artificial General Intelligence or Super Intelligence. I can understand the perspective of investors though... If you can get half of the world on these products at some point you will find something you can sell them even if its not the core product (i.e. loss leader).

[bg24]

AI (llm) will be a commodity market => I am not sure it was obvious. As of last month, folks thought open weight models are lagging by 6+ months. Once K3 is taken for a deep run across many use cases, it will be clear where it stands. But yes, I agree that now that intelligence is commodity, everything changes.

[abernard1]

" - The highest tier Chinese models are not more economical than US frontier models. Try GLM 5.2 and see how much it costs to do real work. I did, and it was more expensive than GPT 5.6."

This is a flatly false statement for most things powering backend applications. The AI consumer "doing real work" model, either for analysis, chat, or coding could well be more cost effective with closed frontier models.

But most of these internal glue business SaaS applications where engineers are integrating are not those tasks. It is those tasks which 1) drive immense amount of domain-specific data into the platform over time, and 2) are most encouraging of driving open model independence with no vendor lock-in.

Anyone on this site who has actually used ML models (more accurate in many cases) knows there's a lot of kludge that simply does not need a 5 minute agentic feedback loop to solve the problem. And they were solvable a year ago with lower class models. The token economics are exceptional and the anecdotes of a16z saying 80% of startups are productionizing open models is only surprising to people who think running your company on OracleDB in 2026 is a sound engineering decision.

[mediaman]

You're correct, but that's a different market segment and not the market GLM 5.2 and its peers compete in.

The labs are not interested in the small, fast, single purpose end of the market. Google increased their pricing on Flash so much that it stopped becoming a cheap model; instead, they released Gemma 4 open source, which is actually easier to use from a third-party inference provider than from Google.

From a total token volume perspective, these "utility" models (classifiers, simple summarizers, small OCR models) will absolutely drive enormous volumes of tokens, at low prices and margin and modest overall market size. Because the models are small and the performance requirements are modest, and because their use cases are specialized rather than general, there are poor economies of scale: they can run cost effectively on rented small GPUs, and a big player doesn't get a structural cost advantage. These models are usually 1b - 30b in size, and can run on a rented 5090. I've productized these myself: I run millions of pages through a fine tuned 1b OCR language model that runs on 5090s at a cost far lower than commercial providers.

But that's not the segment of the market where GLM 5.2, Kimi 3, etc., play. They compete with frontier capabilities, and they are not particularly cheaper than OpenAI models at a cost per task. (I do actually think they compete well with Anthropic, because Anthropic's model efficiencies are poor compared to OpenAI.) And although this part of the market may not be the bulk of the token volume, it is the bulk of the market value.

That's because a lot of human knowledge work is too generalized and fuzzy for dedicated, fine-tuned models, so they are almost entirely different markets that don't particularly compete with each other. (Though if SaaS companies successfully build around verticals that can use small models applied against well-defined jobs, there may be opportunity to push the small/big capability boundary to subsume marginally more valuable tasks that today would require mid-grade reasoning.)

[ignoramous]

> "The highest tier Chinese models are not more economical than US frontier models. Try GLM 5.2 and see how much it costs to do real work. I did, and it was more expensive than GPT 5.6." This is a flatly false statement.

It may not be false but may be a "category error" [0]. Reserved GPU pricing & bulk inference pricing is 3x to 6x cheaper than "API rates", but renting your own GPU cluster (in this crunch) to run a 600b+ open weights is going to be "more expensive than GPT 5.6".

Even then, it remains to be seen if Huawei will pull their weight (and match up to Nvidia) as spectacularly as their fellow Chinese AI Labs have. If so, the WAICO alliance is ready to go all-in.

[0] Ben, and probably other "influencers" in this space, may be prone (knowingly or unknowingly) to favour points that make their conclusion for them (https://en.wikipedia.org/wiki/Motivated_reasoning).

[abernard1]

Fair. Too strong a statement.

But much like Ben's point that commoditization is a relatively novel concept to many in tech, it's not the consumer AI applications at risk of commoditization. They have distribution there.

It's the literally millions of engineers who are updating codebases with tools replacing workers partially or wholly. It's the supply-side where there's compression, and no need for distribution.

I would argue, given the enormity of the existing SaaS stack and how it integrates with the human machinery of personnel, that's where volume is. And that is clearly cheaper and a home run.

Commoditizing a ~$100B AI consumer market is no small feat. Commoditizing 20% of the $500B SaaS market, to say nothing of the underlying systems in the who-knows-how-many trillions "Big Tech" market (you're obligated to say that like the Kool Aid man), is shocking.

[AnthonyMouse]

> Commodity market profitability is determined by marginal cost of production. LLMs have marginal cost; traditional software does not.

This is the story for Nvidia/AMD or cloud providers rather than OpenAI.

> With increasing inference as % of total compute, if labs create efficient models -- which they can, because they can create highly optimized models amortized over very high inference loads -- they can be low cost producers, and be competitive at $/task rates

It seems like there would be problems with this on both ends.

For general purpose models, everybody is trying to make them efficient, so you can't win just by being slightly more efficient. You would have to be so much more efficient that you can charge high margins while still capturing the majority of the market so that the high margins get multiplied by the majority of users and the users you leave on the table aren't funding open competitors. Meanwhile everyone else is also trying to improve efficiency, so one misstep and you're behind.

Example of where this can be a problem: You spend a preposterous amount of money to create an efficient model, then someone else publishes a paper with a new technique that gets a similar but incompatible efficiency improvement out of a model that costs a lot less to create. You have now spent an enormous amount of money in exchange for no competitive advantage.

And from the other end, one of the best ways to get efficiency is through specialization. A general purpose model can generate code or summarize a meeting transcript, but a special purpose model can do it as well or better with far fewer parameters and resources. But then you don't have a situation where one huge AI company has The Most Efficient Model, you instead have dozens of specialized models produced by independent sources that are each the best in a given niche. Any proportion of which could have open weights, or have an arbitrarily small advantage over the ones that are.

Moreover, these problems combine: Both the computing hardware vendors and the AI companies want the margin on doing inference, but the more of it one of them gets, the less the other does. If the AI companies were actually getting huge margins then it would be in the interests of Nvidia, AMD, Apple, Intel et al to fund efficient open weight models in the same way they fund Linux. Commoditize your complement. And those models don't even have to be better, as long as they're good enough that the closed models can't charge a significant premium and the margin shifts back to paying for hardware.

[resonious]

> for frontier work.

I'll agree that GPT 5.6 may well be the best given the above contstraint, but for run-of-the-mill dev tasks (real ones, not benchmark ones), GLM 5.2 still blows every other model out of the water.

Cost per task as a metric is a bit ridiculous because there are so many types of tasks. GPT-5.6 can do some tasks GLM could only dream of, but GLM can do some tasks 100x cheaper and better than GPT-5.6.

[didibus]

China is working on the whole supply chain though and they're willing to compete on razor thin margins. Just look at EVs. They build great cars but the competition is so aggressive that investing in any one Chinese EV company isn't exactly an amazing ROI.

I could see AI ending up the same way where the customer captures most of the value rather than the companies. Open weight models are what make that kind of competition possible.

[marcus_holmes]

> Models are not free. Downloading them is free. Running them is not. This has manufacturing economics, not software economics; the idea that they are "free" is an economic category error as it relates to their actual use

I notice that the article, and this discussion, hasn't mentioned or considered local models.

We can already run a low-spec model on a laptop. Because there is demand for this, it will improve and we will get better laptops and better local models. We will also see models being run on dedicated local hardware and called from the laptop.

If I can download a reasonably capable model to my own hardware and run it without paying anyone for either the model or the inference tokens (effectively making models and intelligence actually free once the hardware is bought) how are the Frontier AI Labs going to make any money at all, let alone enough to support their vast valuations?

[nunez]

This will matter A LOT more after Apple gets serious about integrating AI into the OS.

[marcus_holmes]

Agree, all those consumer ChatGPT accounts will be gone

[gmerc]

Yea but at those rates VCs will never make their money back. Because Deepseek and friends keep releasing the inference optimisations to everyone instead of holding them back to pay their investors.

[tristanj]

I did read the article, but it misses the core issue entirely, and it's why I shared my comment to begin with. Look at the cost-per-task benchmarks from Artificial Analysis https://artificialanalysis.ai/models?cost=cost-per-task

Anthropic’s API pricing is getting impossible to justify. Anthropic previously had the highest quality models, and used their position to charge premium prices, enjoying inference margins of over 70% [0]. They could charge these prices because no other model came close.

But over the past month, the market has shifted dramatically. Over every single performance tier, Anthropic is being squeezed on price.

* Low end: DeepSeek V4 Flash runs at ($0.02/task), Xiaomi's MiMo-V2.5-Pro at ($0.03), and Haiku at ($0.24). Anthropic is ~10x more expensive than the Chinese open-weight options.

* Mid tier: Claude Sonnet 5 ($1.53/task) is nearly 50% more expensive than GPT-5.6 Sol ($1.04), nearly 2x the cost of GPT-5.6 Terra ($0.82), and 3x the cost of GLM-5.2 Max ($0.47). There is basically no reason to ever use Sonnet 5, the competitors are significantly cheaper.

* High end: Opus 4.8 ($1.80/task) and Fable 5 ($2.75) are the two most expensive models, and GPT-5.6 Sol ($1.04) and Kimi K3 ($0.95) offer comparable performance for significantly less. Less the fact that Kimi K3 will get ~10x cheaper once its weights are released and served on neoclouds with Nvidia hardware [1].

OpenAI priced their latest GPT-5.6 models cheaply in order to regain market share. When Anthropic clearly had the best models, their 70%+ inference margins were defensible. But today they are the most expensive option in every single tier. Unless they make significant price cuts soon, they run a serious risk of bleeding market share.

[0] https://www.mindstudio.ai/blog/anthropic-inference-margins-7...

[1] "American companies such as Modal, Fireworks, and Baseten will be able to serve Kimi K3, at one-tenth the cost of their Chinese competitors because they have access to advanced Nvidia hardware" https://x.com/rohanpaul_ai/status/2079027313455550839