[bayindirh]

In short, yes.

A modern laptop is running almost fanless, like a 486 from the days of yore.

A single H200 pumps out 700W continuously in a data center, and you run thousands of them.

Also, don't forget the training and fine tuning runs required for the models.

Mass transportation / global logistics can be very efficient and cheap.

Before the pandemic, it was cheaper to import fresh tomatoes from half-world away rather than growing them locally in some cases. A single container of painting supplies is nothing in the grand scheme of things, esp. when compared with what data centers are consuming and emitting.

[lxgr]

This argument is so flawed that its conclusion almost loops back around to being correct again:

No, in terms of unit economics, I'm almost certain that the painting supplies have a bigger ecological/resource footprint than an LLM per icon generated, and I'm pretty sure the cost of shipping tomatoes does not decrease that footprint, even if it possibly dwarfs it.

But yes, due to Jevon's paradox, the total resource use might well increase despite all that. I, for example, would have never commissioned a professional icon for my silly little iOS shortcuts on my homescreen, so my silly icon related carbon footprint went from exactly zero to slightly above that.

[cpill]

these are unfair comparisons. it's not just a single laptop running all day it's all the graphic designer laptops that get replaced. it's not a single container of painting supplies it's all off them, (which are toxic by the way).

so if power were plentiful and environmental you'd be onboard with it?

[bayindirh]

> these are unfair comparisons. it's not just a single laptop running all day it's all the graphic designer laptops that get replaced. it's not a single container of painting supplies it's all off them, (which are toxic by the way).

Please see my other comment about energy consumption and connect the dots with how open loop DLC systems are harmful to fresh water supplies (which is another comment of mine).

> so if power were plentiful and environmental you'd be onboard with it?

This is a pretty loaded way to ask this. Let me put this straight. I'm not against AI. I'm against how this thing is built. Namely:

    - Use of copyrighted and copylefted materials to train models and hiding under "fair use" to exploit people.
      - Moreover, belittling of people who create things with their blood sweat and tears and poorly imitating their art just for kicks or quick bucks.
    - Playing fast and loose with environment and energy consumption without trying to make things efficiently and sustainably to reduce initial costs and time to market.
    - Gaslighting the users and general community about how these things are built, and how it's a theater, again to make people use this and offload their thinking, atrophying their skills and making them dependent on these.

I work in HPC. I support AI workloads and projects, but the projects we tackle have real benefits, like ecosystem monitoring, long term climate science, water level warning and prediction systems, etc. which have real tangible benefits for the future of the humanity. Moreover, there are other projects trying to minimize environmental impact of computation which we're part of.

So it's pretty nuanced, and the AI iceberg goes well below OpenAI/Anthropic/Mistral trio.

[lxgr]

> I support AI workloads and projects, but the projects we tackle have real benefits [...]

As opposed to the illusory/fake/immoral benefits of using LLMs for entertainment purposes (leaving aside all other applications for now)?

How do you feel about Hollywood, or even your local theater production? I bet the environmental unit economics don't look great on those either, yet I wouldn't be so quick to pass moral judgement.

Why not just focus on the environmental impact instead of moralizing about the utility? It seems hard to impossible to get consensus there, and the impact should be able to speak for itself if it's concerning.

[ToValueFunfetti]

This is a plainly dishonest comparison. A single H200 does not need to run continuously for you to generate a dozen pictures. And then you immediately pivot to comparing the paint usage against "the grand scheme of things"- 700W is nothing in the grand scheme of things.

[bayindirh]

In fact it's pretty fair.

Many people think that when a piece of hardware is idle, its power consumption becomes irrelevant, and that's true for home appliances and personal computers.

However, the picture is pretty different for datacenter hardware.

Looking now, an idle V100 (I don't have an idle H200 at hand) uses 40 watts, at minimum. That's more than TDP of many, modern consumer laptops and systems. A MacBook Air uses 35W power supply to charge itself, and it charges pretty quickly even if it's under relatively high stress.

I want to clarify some more things. A modern GPU server houses 4-8 high end GPUs. This means 3KW to 5KW of maximum energy consumption per server. A single rack goes well around 75KW-100KW, and you house hundreds of these racks. So, we're talking about megawatts of energy consumption. CERN's main power line on the Swiss side had a capacity around 10MW, to put things in perspective.

Let's assume an H200 uses 60W energy when it's idle. This means ~500W of wasted energy per server for sitting around. If a complete rack is idle, it's 10KW. So you're wasting energy consumption of 3-5 houses just by sitting and doing nothing.

This computation only thinks about the GPU. Server hardware also adds around 40% to these numbers. Go figure. This is wasting a lot for cat pictures.

And, these "small" numbers add up to a lot.

[lxgr]

Definitely worth considering in a world in which there are any H200s idling in data centers.

[bayindirh]

Now that's one fine No True Scotsman.

    A: GPUs use a lot of power!
    B: Not all of them are running 100% continuously, eh?,
    A: They waste too much power when they're idle, too!
    C: None of the H200s are sitting idle, you knob!

I mean, they are either wasting energy sitting idle or doing barely useful work. I don't know what to say anymore.

We'll cook ourselves, anyway. Why bother? Enjoy the sauna. ¯\_(ツ)_/¯

[ToValueFunfetti]

B is supposed to be me? I said the H200 doesn't need to be running continuously to generate a dozen images. If a million people generate a dozen images, it no longer makes sense to compare to the costs of a single artist for 6 hours. I really don't understand why this is hard and that makes this feel very uncharitable.

[lxgr]

I'm not saying that this isn't "true idling", I'm saying that idling H200s simply don't exist, i.e., I disagree with B. Do you, A, even disagree?

> they are either wasting energy sitting idle or doing barely useful work

Now here's a true (inverse) scotsman, or more accurately, a moved goalpost: Work on things you don't deem valuable is basically the same thing as idling?

> We'll cook ourselves, anyway. Why bother? Enjoy the sauna. ¯\_(ツ)_/¯

I'm very concerned about that too, but I don't think we'll avoid the sauna with fatalism or logically unsound appeals to morality about resource consumption.