[pier25]

Inference is probably the cheapest part of it all.

The buildings will last for years, decades even, but the GPUs need to be replaced every 2-3 years in perpetuity.

[FabCH]

True, but only because they are choosing to actively burn VC money on training.

But it’s not like the models we have now would stop to exist if training stopped. Other than the occasional retraining to get the latest data in, if they stopped wanton experimentation with models, that admittedly is pushing the models forward, the training costs could plummet and inference would be the thing to optimize and scale.

[pier25]

Without new models how would AI companies compete then?

Other features can easily be copied (eg Claude Code, ChatGPT, etc).

If AI services are a commodity then they can only compete on price.

[FabCH]

Well, yeah. Which is where energy efficiency of FPGA and ASIC comes into play.