[knotimpressed]

At least a couple years ago, a big advantage of Nvidea cards was how much cheaper they were to run power wise-often the dies that made it into cloud level cards would be binned consumer dies.

Not sure if that’s still the case, but I’d say it’s plausible.

[lostmsu]

Impossible. Power costs for H100-like cards are dwarfed by the cost of the cards themselves. H100 at full load will consume ~$3500 (rough estimate) of power in 5 years at $0.12/kWh.

[yencabulator]

Data centers are more constrained by availability of power, and the matching cooling, than the actual bulk cost of it.

For example, I've been in situations where we had to deploy fewer hard drives per server unit than we otherwise would have, just because we knew we couldn't power & cool the racks if we fully stocked them.

[lostmsu]

How much do you think it would cost to install cooling for 5kW power output (1 8xH100 node)?

[yencabulator]

It's more that a room/wing/building/whatever is built with a certain capacity, while power and heat density keeps increasing. Most buildings can't "just add cooling for 5kW". Newer data centers should already be built with that lesson in mind, for example with a lot of extra air moving capacity etc planned in, but that's been the struggle for a while now.