[michaelt]

> Power is only a small part, single digit, it will eventually matter but does not really today.

Sorta yes, sorta no.

A single 5090 consumes 450W - at Californian energy prices of $0.38 per kWh that's $0.17 per hour. And the card itself costs $4100 on amazon. So after 2.75 years running at full power 24/7 you'll have spent more on electricity than on the card. I would have thought most data centres being built today would have a design life longer than 3 years.

Of course you can throttle the cards to ~300W without losing too much performance. But also you need more than a single 24GB card to run most modern LLMs.

[verall]

DC has wider margins than the 5090. by someone elses rough numbers (https://www.spheron.network/blog/gb300-nvl72-vs-gb200-nvl72-...) for GB300 rack:

> 132kW

> ~$3.7-4M

So about 300x the 5090's power but 1000x the price. Roughly 9 years for electricity to exceed price at $0.38 and datacenters will show up in areas with cheaper power than CA.

[Omniusaspirer]

Anyone seriously building out AI infrastructure I presume is paying nowhere near $.38/kWh which is extortionate. Utility scale solar is closer to $.02-.03/kWh, then maybe around ~$.10/kWh for natural gas peaker plants.

[londons_explore]

Utility scale power price varies widely by location and exact time of day.

LLM's aren't very latency sensitive and can therefore move to wherever power is cheapest.

Right now that's places next to aluminium smelters (which also like very cheap electricity 90+% of the time).

[inigyou]

GPU design life is commonly cited as about 3 years. The main driver is not physical degradation of the cards but the expectation they'll become obsolete with newer cards.

[threatripper]

Also add electricity 50% on top for cooling the DC.