[Tycho]

What’s the energy profile of running inference in a typical ChatGPT prompt compared to:

  - doing a google search and loading a linked webpage
  - taking a photo with your smartphone and uploading it to social media for sharing
  - playing Fortnite for 20 minutes
  - hosting a Zoom conference with 15 people
  - sending an email to a hundred colleagues

I’d be curious. AI inference is massively centralised, so of course the data centres will be using a lot of energy, but less centralised use cases may be less power efficient from a wholistic perspective.

[JimDabell]

A ChatGPT prompt uses 0.3 Wh, which is approximately how much energy a Google search took in 2009.

AI energy use is negligible compared with other everyday activities. This is a great article on the subject:

https://andymasley.substack.com/p/a-cheat-sheet-for-conversa...

The same author has published a series of articles that go into a lot of depth when it comes to AI energy and water use:

https://andymasley.substack.com/p/ai-and-the-environment

[danans]

> A ChatGPT prompt uses 0.3 Wh, which is approximately how much energy a Google search took in 2009.

That's the number their CEO put out, but AFAIK it is completely unverified (they did not provide any background as to how it was calculated). To believe it is an article of faith at this point.

What is concrete and verifiable are the large deals being struck between AI model providers and energy providers - often to be supplied via fossil fuels.

[JimDabell]

> That's the number their CEO put out, but AFAIK it is completely unverified (they did not provide any background as to how it was calculated). To believe it is an article of faith at this point.

Google also puts the median Gemini prompt at 0.24 Wh. The information available from different sources point in the same direction; you don’t have to take Sam Altman’s word for it. 0.3 Wh was the figure that was already pretty dependable before he said that.

> What is concrete and verifiable are the large deals being struck between AI model providers and energy providers - often to be supplied via fossil fuels.

Which is completely irrelevant to this discussion unless you quantify that in Wh per prompt. Vague “deals are being struck!” hand-wringing doesn’t add to the discussion at all. Why are you demanding absolute, unimpeachable rigour when vendors give specific figures, but are comfortable with hand waving when it comes to complaining about energy use?

[danans]

> Why are you demanding absolute, unimpeachable rigour when vendors give specific figures, but are comfortable with hand waving when it comes to complaining about energy use?

Because the evidence that AI data centers are using a lot of energy (generated by fossil fuels in particular) is observable in current concrete reality, like the xAI datacenters running gas turbines adjacent to a neighborhood in Memphis, TN:

https://gasoutlook.com/analysis/xai-data-centre-emits-plumes...

Companies like OpenAI, Google, and xAI have an incentive to downplay the energy usage of their facilities. If not, they should publish their methodology. The burden is on them to prove that they won't drive up energy prices and increase emissions.

[JimDabell]

> AI data centers are using a lot of energy

This is a worthless thing to say in the context of this discussion.

How much is “a lot”? How many queries does that service?

Saying that “data centres use a lot of energy” doesn’t tell us anything at all about their energy efficiency.

> Companies like OpenAI, Google, and xAI have an incentive to downplay the energy usage of their facilities. If not, they should publish their methodology.

All you’re really doing here is giving me the impression you aren’t participating in this discussion honestly. You demand more detail, but if you had read the things I pointed you towards, you would see that Google have published what you want already. You don’t actually care about the details, you want to complain without bothering to look at the information handed to you on a plate.

[slfnflctd]

These are the kinds of questions we need pursued to develop better insight into the overall societal impact of current and near-future LLMs. Energy usage is a critical measure of any technology. The tradeoffs between alternate use cases should be modeled as accurately as possible, including all significant externalities.