[confuseshrink]

The Strubell paper which is the origin of this "5 cars" number isn't even in the right ballpark for this stuff. What they did was take desktop GPU power consumption running the model in fp32, extrapolate to a 240x GPU (P100) setup that would run for a year straight at 100% power consumption.

Yes if you do run 240x p100s at literally 100% 24/7 for a year you get the power consumption of 5 cars. This run never happened though, this all ran on TPUs at lower precision, lower power consumption and much lower time to converge.

If anything this tells you that electronics are ridiculously green even when operating at 100%. I've never profiled world-wide carbon production but something tells me if you wanted to carbon optimise you'd be better served trying to take cars off the road and planes out of the sky.

[the-rc]

As I already mentioned, the paper also uses an industry average PUE factor of 1.58, when Google's is 1.1. Other large tech companies can't be too far behind.

Which makes me wonder how far removed AI researchers are from actual production environments. I'm not faulting them, because there's only so much time in your life; the more realistic problem is when someone else takes a paper as gospel and runs headlines with it. Kinda like the trolley problem. Imagine the absurd extreme in this case of governments wanting to regulate large language models because of pollution or to level the competition playing field.

[CydeWeys]

> I've never profiled world-wide carbon production but something tells me if you wanted to carbon optimise you'd be better served trying to take cars off the road and planes out of the sky.

We're getting a bit off-topic here, but the #1 target by far in reducing greenhouse emissions is power generation. In transportation it's significantly trickier to replace petrol-based fuels (especially for airplanes), but it's straightforward enough in power plants. And crucially, you can convert all the petrol-powered vehicles to EVs that you want, but if the electricity they're getting from the wall is still provided by burning petroleum then you haven't actually done that much.

Luckily, computation can for the most part be located anywhere (exactly the opposite of transportation), and thus you have a lot of data centers near hydro and other renewable sources so that they can use the cheapest green power available.

[confuseshrink]

> We're getting a bit off-topic here, but the #1 target by far in reducing greenhouse emissions is power generation.

I readily admit I don't know any of the numbers associated with carbon production and my comment was solely based on the one GPU vs car figure presented in the aforementioned paper.