[lovemenot]

Isn't it the case that training a ML model consumes vastly more power than using the trained model for inference?

If so, the problem may be somewhat overstated. Training is a dispatchable power load. It can be scheduled for periods of surplus power in regions where surplus power is periodically abundant. Training a ML model could literally follow the sun.

[brucethemoose2]

At Big Tech scale, the same model is used for inference so many times that most of the energy is probably from inference.

[palata]

Note that if you have surplus of power, maybe you want to use it more wisely :).

[lovemenot]

Perhaps, you are right about utility, but I can think of few power loads as well suited to dispatch.

Bauxite has to be shipped to Iceland to take advantage of their abundant geothermal and hydro power for Aluminium smelting. Data moves with much less friction.