[killerstorm]

A continuously running agent might require 100-500W for inference, so comparable to gaming or a small space heater. Not obscene, but also not negligible.

If we assume 250W for a continuously running agent, Grok 4 training run estimate would be around 50 million session-days, so a half-million people might consume as much running agents continuously for 100 days.

[roenxi]

We might suspect the hardware used for the LLM is very similar to a graphics card and so the wattage would be extremely similar to using a graphics card to display a game's graphics in the first place. Which puts a certain ironic perspective on tokarf's original comment.

[killerstorm]

To serve a frontier model you need a lot more than one GPU... at least 8. But for better efficiency they batch multiple requests on one server. So it is sort of a coincidence that it ends up roughly in 1 consumer GPU range. A different architecture (e.g. linear attention / RNN) could end up with more peak power usage.