[joefourier]

Outside of training the biggest LLMs at big labs, GPU lifespan isn't as short as the OP made it out to sound. A100s are 6 years old and still a reliable work-horse, and the 80GB version hasn't depreciated that much on the used market. On the consumer side, 3090s are actually still selling for very close to 2020 MSRP.

Even the ancient V100 (soon to be 10 years old!) had somewhat of resurgence on the second-hand market, with a healthy market for interconnects in China.

If I had a datacenter and power consumption was not a concern, I'd be holding on to my A100s for years at least for inference.

[throwawayffffas]

Oh yeah, not meant to be all doom and gloom. Lighter workloads greatly increase hardware lifespan. And the GPUS are like at most 50% of the data-center cost I think. You get to keep the building, the cooling, the power interconnects, the networking and everything else.

Additionally the demand drives new power infrastructure, and new fabs that will definitely outlive the bubble.