[smodad]

What's funny is that even though the DGX GH200 is some of the most powerful hardware available, there's such a voracious demand that it's not gonna be enough to quench it. In fact, this is one of those cases where I think the demand will always outpace supply. Exciting stuff ahead.

I heard Elon say something interesting during the discussion/launch of xAI: "My prediction is that we will go from an extreme silicon shortage today, to probably a voltage-transformer shortage in about year, and then an electricity shortage in about a year, two years."

I'm not sure about the timeline, but it's an intriguing idea that soon the rate limiting resource will be electricity. I wonder how true that is and if we're prepared for that.

[jiggawatts]

He’s just plain wrong about the electricity usage going up because of AI compute.

To a first approximation, the amount of silicon wafers going through fabs globally is constant. We won’t suddenly increase chip manufacturing a hundredfold! There aren’t enough fabs or “tools” like the ASML EUV machines for that.

Electricity is used for lots of things, not just compute, and within compute the AI fraction is tiny. We’re ramping up a rounding error to a slightly larger rounding error.

What will increase is global energy demand for overall economic activity as manufacturing and industry is accelerated by AIs.

Anyone who’s played games like Factorio would know intuitively that the only two real inputs to the economy are raw materials and energy. Increases to manufacturing speed need matching increases to energy supply!

[smodad]

I bet you're right. Even if you take into account that a data center is a monster consumer of energy, in the grand scheme of things it's not that big. Some back of the envelope math:

Global electrical production in 2022 was ~30,000 TWh.[1]

If we over-estimate that a hyperscale data-center will consume about 100 MW of power, per year that would be around 876 GWh.[2]

Let's overestimate again and say that 1,000 new data centers spring up in a year, every year they would consume 876 TWh.

Which, is 2.92% of total electricity production. Which given the fact that I overestimated the energy consumption by more than an order of magnitude, I would say the term "rounding error" is accurate.

I think the main limiting factor in the near term is going to be chip production capacity. The fabs take so long to spin up, it's going to be a while before we can even consider "electricity production" being a limiting factor.

[1] https://yearbook.enerdata.net/electricity/world-electricity-... [2] https://cc-techgroup.com/data-center-energy-consumption/

[paskjdfparwerwe]

Elon is speaking with all the Eliezur-esque "foom" in mind, where in AI will explode and either kill us or help us take over the Universe (and destroy everything in our way).

[wmf]

A wafer of H100s uses far more electricity than a wafer of [Apple] A16s though.

[pclmulqdq]

I'm pretty sure that's actually wrong. Some math:

A16 is 200 sq mm of silicon while an H100 is about 800. That means you get about 100-120 A16's on a wafer, while you only get ~30 H100's (see https://www.silicon-edge.co.uk/j/index.php/resources/die-per...).

Let's assume yield is 100% to make things easier. The rated max power of the A16 is about 250W, while the H100 is quoted at 700W. Thus, a wafer of A16's is about 25-30 kW of power, while a wafer of H100's is about 21 kW.

Edit: Just clarifying, this is not about the Apple A16, but the Nvidia A16. The mobile process used by the Apple chips is built for much lower performance and power, so I can't imagine the two chips being anywhere near comparable - they fill two completely different roles.

[wmf]

That's my point; if silicon demand shits from mobile to AI data centers you can't expect energy consumption to be the same.

[pclmulqdq]

Demand right now is not shifting from mobile to datacenter, demand is shifting from "normal" datacenter compute to AI datacenter compute.

I think if you had said "AMD Epyc" rather than a mobile chip, that would be a much more apt comparison. The AI chips are somewhat more power intensive per box, but fairly similar on power/area. It turns out that these silicon processes are fairly uniform in terms of the power/area that they can sustain for any kind of workload.

Mobile chips are designed for <10% utilization and "rush-to-idle" workloads, and they are not remotely comparable to datacenter silicon (of any kind).

[lwneal]

An H100 uses up to 350 Watts, while an A16 has a TDP of only 8 W. But, the A16 is a smaller chip (about 108mm vs. the H100's 814mm) so you can fit more of them on a wafer. Since a wafer is 300mm in diameter, its area is 70685 mm^2, which would yield 86 H100's or 654 A16's. [1][2]

However, that discounts the waste on the edges of the circular wafer, as well as the chip yield, which will both likely be worse for the larger chip [3]. But, assuming a generous 70% yield by area [4], one wafer's worth of H100s all packaged into GPUs and running full blast will use maybe 20 kilowatts, while the same wafer of A16s might use 3.6 kilowatts. Although in practice, the A16s will spend most of their time conserving battery power in your pocket, and even the H100s will spend some of their time idle.

TSMC is now producing over 14 million wafers per year. At most 1.2 million of those are on the 3nm node, and not all of that production goes to GPUs. But as an upper bound, if we imagine that all of TSMC's wafers could be filled up with nothing but H100 chips, and if all of those H100 chips were immediately put to use running AI 24/7, how much additional load could it put on the power grid every year?

The answer is, around 280 gigawatts, or if they were running 24/7 for a year, about 2500 terawatt-hours. That's about 10% of current world electricity consumption! So it's not completely implausible to imagine that a huge ramp-up in AI usage might have an effect on the electric grid.

*edit: This assumes we're talking about the Apple A16 (ie. the difference between phone chips and GPU chips). If we're talking about the Nvidia A16 (ie. the difference between current GPU chips and last node's GPU chips) see pclmulqdq's comment. ⠀

[1] https://nanoreview.net/en/soc/apple-a16-bionic

[2] https://www.techpowerup.com/gpu-specs/h100-pcie-80-gb.c3899

[3] https://news.ycombinator.com/item?id=24185108

[4] https://www.extremetech.com/computing/analyst-tsmc-hitting-5...

[5] https://www.tsmc.com/english/dedicatedFoundry/manufacturing/...

[6] https://www.wolframalpha.com/input?i=%2814+million%29+*+%282...*

[pclmulqdq]

8 watts for the A16's TDP cannot be correct. Your phone CPU has a higher TDP. I saw 250 on Nvidia's website as a maximum.

Edit: Oh, you are talking about the Apple A16. Those chips are completely different in function, so sure.

[adrian_b]

6 to 8 W is a typical TDP for a mobile phone SoC including the CPU.

A few mobile phone chips had a higher TDP, up to 10 W, but those were notorious for overheating and for low battery life.

[blackoil]

> At most 1.2 million of those are on the 3nm node

1.2 x 30 x 30000($/board) ~ 1 trillion $$$. Time for NVDA call.

[michaelt]

> I wonder how true that is

An Nvidia A100 costs $10000 and consumes 300W.

It seems unlikely that anyone could afford the number of A100s needed to create an electricity shortage.

If there is an electricity shortage, far more likely that ageing infrastructure and rising demand for air conditioning and electric car charging are to blame.

[callalex]

Are there any examples at all about that guy being right about a technology prediction?

[kanwisher]

Electric cars, rocket ships …

[losteric]

Elon's timeline predictions in both of those industries for his own companies have been consistently wrong for years. (FSD when??)

Given we're talking about hardware for software, let's at least look at his track record in the software industry... glances at Twitter ah, yeah, not great either.

Voltage regulator and electricity shortage from AI growth straight up doesn't make sense, it's dumber than the stuff he was spouting while "deep-diving" his Twitter misacquisition.

[swyx]

i mean he's not the only one. sama's other big bet is on nuclear fusion. https://blog.samaltman.com/helion