[mlyle]

> So long as there is competition it’ll be available at marginal cost.

Most things are not perfect competition, so you get MR=MC not P=MC.

We're talking about massive capital costs. Another name for massive capital costs are "barriers to entry".

[cjbgkagh]

Granted that capital costs are a barrier to entry and that barriers to entry leads to non-perfect competition, but the exploitability is limited in the case of LLMs because they exist on a sub-linear utility scale. In LLMs 2x the price is not 2x as useful, this means a new entrant can enter the lower end of the market and work their way up. The only way to prevent that is for the incumbent to keep costs as close to marginal as possible.

There is a natural monopoly aspect given the ability to train and data mine on private usage data but in general improvements in the algorithms and training seem to be dominating advancements. Microsoft's search engine Bing paid an absolute fortune for access to usage data and they were unable to capitalize on it. LLMs have the unusual property that a lot of value can be extracted out of fine tuning for a specialized purposes which opens the door to a million little niches providing fertile ground for future competitors. This is one area where being a fast follower makes a lot of sense.

[mlyle]

Almost anything has a utility scale which is diminishing. But we still see MR=MC pricing in industries with barriers to entry (IPR, capital costs). TSMC and Mercedes don't price cheap to avoid giving others a toehold.

> There is a natural monopoly aspect given the ability to train and data mine on private usage data but in general improvements in the algorithms and training seem to be dominating advancements.

There's pretty big economies of scale with inference-- the magic of how to route correctly with experts to conduct batching while keeping latency low. It's an expensive technology to create, and there's a large minimum scale where it works well.

[cjbgkagh]

I’m unconvinced that the lessons learned from scaling will constitute much of a moat. There is certainly an incentive for incumbents to give such an impression.

[mlyle]

Probably not. But we don't tend to see P=MC where there's any differentiation or barrier to entry, and I do not believe that AI is fully commoditized or will be soon.

[cjbgkagh]

Perhaps my 'marginal cost' should have been taken to be 'near marginal cost' and as such I don't believe it'll be fully commoditized either, just mostly commoditized in that it becomes impossible to extract meaningful monopolistic rents. Similarly I don't believe in perfect market efficiency so nothing would be exactly at marginal cost. At the moment we have investment subsidizing use so often these APIs are available at below marginal cost - the old adage 'we lose money on every sale but make up for it in volume'. I'm not confident that these investors on average will be able to make their money back plus a required rate of return and for many it's probably not the primary point of investing in this industry. If you have a $1B dollars to spare you too can light it on fire...

[mlyle]

I don't believe the APIs are actually under marginal cost. Inference is cheap, but people are using a lot of it.

The problem is, the arguments you're making could be used for almost any industry, including ones that we've seen sustained excess profits.