[malfist]

Each of these GPUs pull up to a kilowatt of power. The average commercial power cost is 13.4 ¢/kWh. That means running a single H100 full tilt 24/7 is a power operationing cost of $1,100 per card per year.

In three years the current generation of GPUs will be 50% or more faster. In six years your talking more than 100% faster. For the same energy costs.

If you're running a GPU data center on six year old GPUs, your cost to operate per sellable unit of work is double the cost of a competitor.

[Ekaros]

One thing I am not entirely sure if there will be huge efficiency gains. Just looking at TDP that is the power consumption of say 3090 and 5090 and the increase is substantial then compare it to performance and the performance lift stops looking that great...

[xyhopguy]

3x increase in compute for a 1.5x increase in tdp is pretty good considering the underlying process had barely changed. In anycase, consumer GPUs aren't a good metric as they operate with different economic constraints.

H100 to GB200 saw a 50x increase in efficiency, for example.

[Ekaros]

https://www.nvidia.com/en-us/data-center/gb200-nvl72/

Nvidia only advertises 25x efficiency. And that is their word...

[brookst]

Sure. But if that fully depreciates, $1100/year GPU produces $20k of economic benefit, would you decommission it as long as there is demand?

[malfist]

If my data center sells a pflop at $5 because of our electricity use and the data center a state over with newer GPUs sells it at $2.50/pflop, it doesn't matter how much economic benefit it generates, my customers are all going to the data center a state over.

[voakbasda]

I want to see math on how a single GPU will pull down that much revenue, because that seems like a dubious outcome.

[brookst]

Fair, I was hand waving to make a point. “If it generates more than $1100 + (resale price * WACC) + opportunity cost from physical space/etc” would have been more accurate.

But the point is — you don’t decommission profit generators just because a competitor has a lower cost structure. You run things until it is more profitable for you to decommission them.

[malfist]

That all depends on if you're running your own hardware (unlikely) or renting.