[bob1029]

I'd be very interested in seeing some kind of aggregated daily demand curve for AI workloads.

It seems like a lot of the hyperbolic angles are looking at this as a constant draw of power over time. There is no reason for a GPU inference farm to be ramped up to 100% clock speed when all of its users are in bed. The 5700XT in my computer is probably pulling a mere 8~12W right now since it is just sitting on an idle desktop. A hyperscaler could easily power down entire racks based upon anticipated demand and turn that into 0W.

[pixl97]

With the current demand for GPU time it makes no sense to have your units idle. If you're not selling infrence to customers you'll batch out training to whoever will rent the excess time from you.

Maybe in the future there will be idle time.

[datadrivenangel]

The point is that training activity has cost more than the marginal cost of running the hardware, if not, it's better to keep it idle.