[acchow]

> This is a question that analysts don't even ask

On the contrary, data centers continue to pop up deploying thousands of GPUs specifically because the numbers work out.

The H100 launched at $30k GPU and rented for $2.50/hr. It's been 3 years since launch, the rent price is still around $2.50.

During these 3 years, it has brought in $65k in revenue.

[kd913]

They worked out because there was an excess of energy and water to handle it.

We will see how the maths works out given there is 19 GW shortage of power. 7 year lead time for Siemens power turbines, 3-5 years for transformers.

Raw commodities are shooting up, not enough education to cover nuclear and SMEs and the RoI is already underwater.

[SketchySeaBeast]

My cynical take is that it'll works out just fine for the data centers, but the neighbouring communities won't care for the constant rolling blackouts.

[kd913]

Okay but even in that case the hardware suffers significant under utilisation which massively hits RoI. (I think I read they only achieve 30% utilisation in this scenario)

[SketchySeaBeast]

Why would that be the case if we assume the grid prioritizes the data centers?

[kd913]

That is not a correct assumption. https://ig.ft.com/ai-power/

Reports in North Virginia and Texas are stating existing data centres are being capped 30% to prevent residential brownouts.

[SketchySeaBeast]

That article appears to be stuck behind a paywall, so I can't speak to it.

That's good for now, but considering the federal push to prevent states from creating AI regulations, and the overall technological oligopoly we have going on, I wonder if, in the near future, their energy requirements might get prioritized. Again, cynical. Possibly making up scenarios. I'm just concerned when more and more centers pop up in communities with less protections.

[fauigerzigerk]

Beyond GPUs themselves, you also have other costs such as data centers, servers and networking, electricity, staff and interest payments.

I think building and operating data center infrastructure is a high risk, low margin business.

[hdjrudni]

They can run these things at 100% utilization for 3 years straight? And not burn them out? That's impressive.

[imtringued]

I don't see anything impressive here?

[vlovich123]

Not really. GPUs are stateless so your bounded lifetime regardless of how much you use them is the lifetime of the shitties capacitor on there (essentially). Modulo a design defect or manufacturing defect, I’d expect a usable lifetime of at least 10 years, well beyond the manufacturer’s desire to support the drivers for it (ie the sw should “fail” first).

[mike_hearn]

The silicon itself does wear out. Dopant migration or something, I'm not an expert. Three years is probably too low but they do die. GPUs dying during training runs was a major engineering problem that had to be tackled to build LLMs.

[Majromax]

> GPUs dying during training runs was a major engineering problem that had to be tackled to build LLMs.

The scale there is a little bit different. If you're training an LLM with 10,000 tightly-coupled GPUs where one failure could kill the entire job, then your mean time to failure drops by that factor of 10,000. What is a trivial risk in a single-GPU home setup would become a daily occurrence at that scale.