[bogtog]

A few months ago, there was a lot of news lambasting tech companies for extending the depreciation lifespan of GPUs from ~3 years to ~5 years. Do these price hikes suggest a longer lifespan is probably the right way to see how long these GPUs will be valuable?

[miningape]

Not a finance guy, so fully prepared to be wrong here. But my interpretation is that an increase in price corresponds to a shorter lifespan. i.e. less time to make money so we need to charge more to get the same return over less time

It could also be a supply/demand issue, generally price increases are caused by either 1. demand increasing, or 2. supply decreasing.

In this case we can interpret a shorter lifespan as decreased supply, but it can also be because the demand for GPU compute has gone up. I think in this case we're seeing a bit of both, but it's hard to tell without more data.

We could also consider the supply / demand elasticity changing, f.x since demand has become more price inelastic it could result in a higher price.

[lukeschlather]

The thing historically about GPUs has not been the actual lifespan of the hardware (at least half of the hardware will probably work fine for 10 or more years) the problem is that work/watt is dropping for newer hardware, so there's a point where even if you had an equivalent quantity of 10-year-old GPUs, powering them for some period costs $40k and you can buy a single brand-new GPU that costs $40k but only costs $20K to power for the same period which is less than a few years.

I don't think we're seeing any decrease in supply though, ignoring 2020 I'm pretty sure the number of GPUs manufactured has been steadily increasing. It might be the case that projected manufacturing was higher than what actually happened, which is not the same thing as a decrease in supply, but companies like Amazon will talk about it like it is, and from the standpoint of their pricing it essentially is.

[zozbot234]

> the problem is that work/watt is dropping for newer hardware, so there's a point where even if you had an equivalent quantity of 10-year-old GPUs, powering them for some period costs $40k

Sell the old-gen GPU's to on-prem users (including home consumers) who are going to run them a small % of the time (so power use is more or less negligible to them compared to acquisition cost), problem solved.

[lukeschlather]

The same math applies for on-prem/home users. If you actually have some workload where it makes sense to get a free GPU that costs $40/hour to power because you only need it for a few hours a month, it's probably cheaper to rent a more efficient GPU from someone who can power it at a lower cost.

[bogtog]

Oh my reasoning was coming at this from a different angle: H200s were released in November of 2023, so they're over 2 years old at this point while still being valuable

[lukeschlather]

I would not be surprised if the majority of H200s were manufactured in the past 12 or even 6 months.

[xiphias2]

It mainly depends on how much NVIDIA is overselling the improvements.

With adding RL functions, separating prefill and decode chips, nvfp4 and lots of other architectural changes efficiency of the most valuable tasks goes up as long as the algorithms don't change significantly.

Everything else can just stay on older chips.

[Der_Einzige]

The people who think that high end GPUs don't last for ~5 years (really it's 6) do not know what they are talking about. 6 years is likely too low. If the cooling is good and they don't fail early, most of these GPUs could still keep going past 6 years.

I'm convinced that anything with more than 80gb of VRAM will be worth it for closer to 10 years at this point.