[elorant]

Even if we assume that everything you said holds true, how is that we as a crowd can make viable a service that eats some $300bn annually in infrastructure costs? Where would that money come from? Most tech companies these days are cutting their AI budgets because the per token pricing is killing them.

[aspenmartin]

Cite a real source for that last bit, I don’t think that is true. Also the budgets should be cut the spend at some places goes beyond any reasonable amount. The strategy there is to hook everything in and find the right processes, then cut the rest. Things then get better and better with each model release.

The way you make a viable service that eats 300bn annually is to have enough demand to service that. Anthropic underbought compute. That tells you something.

[elorant]

https://www.theverge.com/tech/930447/microsoft-claude-code-d...

https://finance.yahoo.com/sectors/technology/articles/ai-bin...

https://blog.pragmaticengineer.com/the-pulse-token-spend-bre...

[jazzcomputer]

When you say "Things then get better and better with each model release."

How far behind are models that can be run locally, and do you expect that this will be widespread?

[aspenmartin]

https://epoch.ai/data-insights/consumer-gpu-model-gap

I think over the years local models have fairly consistently been ~7 months behind frontier performance. Local models are hugely important but I don’t see the calculus changing. I can imagine it’s certainly the case for many tasks that there will be diminishing gains for performance improvements or reliability pass some threshold, in which case you don’t need frontier performance and you can certainly use local models or at least cheaper tiers of proprietary models if local is too much of a hassle. Plus of course use cases where local is necessary or the pros of having local models or on device models outweighs that of frontier.

Maybe things will change though, I would assume through basically government subsidies from China etc, to undercut existing frontier labs, but you can always spend more (better data more compute etc) for better performance and that I can imagine will always have a selling point.