A continuously running agent might require 100-500W for inference, so comparable to gaming or a small space heater. Not obscene, but also not negligible.
If we assume 250W for a continuously running agent, Grok 4 training run estimate would be around 50 million session-days, so a half-million people might consume as much running agents continuously for 100 days.
We might suspect the hardware used for the LLM is very similar to a graphics card and so the wattage would be extremely similar to using a graphics card to display a game's graphics in the first place. Which puts a certain ironic perspective on tokarf's original comment.
To serve a frontier model you need a lot more than one GPU... at least 8. But for better efficiency they batch multiple requests on one server. So it is sort of a coincidence that it ends up roughly in 1 consumer GPU range. A different architecture (e.g. linear attention / RNN) could end up with more peak power usage.
All of a sudden we are selectively squeamish with computer resource usage, when we were fine having all that fun with computers and hardware, 3 monitor setups, using graphic cards to play games (dear lord!) and tinkering around with home rigs of every proportion and wattage for no reason at all.
Driving a car consumes 25+ more energy per hour than gaming. So urban planning which encourages people to drive likely results in an order of magnitude more waste than all home computer use.
Presumably you're fine with using electricity to heat/cool your house, yes?
Would you, then, also be fine with running three extra air conditioners/space heaters that do nothing but sit outside on the lawn?
"People are OK with the energy usage already happening, so they should be OK with adding 32,767 AI datacenters too" doesn't make sense. It's essentially a reductio ad absurdum. Of course we're "selectively squeamish with computer resource usage"; some usage is obviously useful to us (with entertainment also in the "useful" category—we're not robots!), while for many, many of us, the extra datacenters are having anywhere from a neutral to a profoundly net-negative impact on our lives even before you consider the resource usage.
I mean, if you actually understood the scale of AI power usage, it would make sense. A TV can run 200 watts/hour, so this is equivalent to... watching TV?
Or, perhaps, watching TV in the background while you program.
We are not in a power crisis, and data centers have had basically no impact on our lives compared to previous technologies. There are more data centers used for the internet, but somehow you're still here, using a data center to complain about checks notes data centers? Really? Well that's a bit cheeky.
Sorry I'm a bit old school I don't watch tv when I program. What's the point of watching tv while programming anyway ? But can you really watch tv while programming ?
I can't
But maybe when you watch tv while programming you can watch documentaries about water crisis triggered by I datacenter usage, or maybe you can watch some reports about Elon using very poluting gas turbines to generate energy for their massive grok datacenter ...
I'm sorry, when individual datacenters are drawing more power than entire cities, and needing to build entire new fossil-fuel-burning power plants to service their needs, you'll excuse me if I'm skeptical of your "it's just like watching TV" excuse there.
You might find it enlightening to look at the power draw and environmental impact of other technologies, for a sense of scale.
LLMs didn't even exist when we started expressing concern about global warming, the internet still uses the vast majority of data centers and thus power, smartphone manufacturing is a nightmare on multiple levels, the environmental effects of factory farming are staggering even if you apply zero moral weight to the suffering, the existence of automobiles... what automobiles used to be like...
It just seems bizarre to care about these issues solely in the context of one particular technology, while giving a free pass to everything prior.
No one's saying we shouldn't care about the impact of other technologies and industries too.
But when the AI industry is sinking hundreds of billions of dollars into building a huge number of datacenters, many of which have already been proven to have environmental and legal problems, sitting there saying, "You didn't complain when we got TV!" or even "You didn't complain when the datacenters were for AWS!" is just transparently trying to change the subject without addressing the very real concerns.
If we assume 250W for a continuously running agent, Grok 4 training run estimate would be around 50 million session-days, so a half-million people might consume as much running agents continuously for 100 days.