[symbolicAGI]

The frontier models when released are operating UIs and APIs at a substantial profit during the delivery of inference. However, overall the vendors are losing money because they are paying for ever-increasing training costs for the next version of their frontier model.

This money-losing business of the vendors will no doubt continue for at least another year.

There are two ways to expect lower LLM API costs in the future:

1. Be satisfied with an older version of a particular LLM. As inference hardware and software become more efficient, the vendor can lower API costs on the older models to remain competitive.

2. Eventually - not next year - the return on investment from training the next version of the LLM will decrease relative to the ROI on current LLMs (because the improvements will be less awesome) and the training cost of such a model will necessarily be spread out over a longer duration as competition allows. At that point (whenever) the training cost might level off or actually decrease and that savings would be competitively passed along to the API consumer. And coincidentally that would be the point at which the vendors become overall profitable.

[changisaac]

This is a great analysis btw, thanks for this!

My take away from this is that my startup should spend some time investing in some cost analysis with our LLM usage and context engineering (perhaps closely after some level of PMF). If it’s not happening anytime soon, might as well treat it as it’s not happening at all considering that startups die out pretty quick lol.