[elbasti]

I work in the datacenter space. The power consumption of a data center is the "canonical" way to describe their size.

Almost every component in a datacenter is upgradeable—in fact, the compute itself only has a lifespan of ~5 years—but the power requirements are basically locked-in. A 200MW data center will always be a 200MW data center, even though the flops it computes will increase.

The fact that we use this unit really nails the fact that AI is basically refining energy.

[aurareturn]

  A 200MW data center will always be a 200MW data center, even though the flops it computes will increase.

This here underscores how important TSMC's upcoming N2 node is. It only increases chip density by ~1.15x (very small relative to previous nodes advancements) but it uses 36% less energy at the same speed as N3 or 18% faster than N3 at the same energy. It's coming at the right time for AI chips used by consumers and energy starved data centers.

N2 is shaping up to be TSMC's most important node since N7.

[alberth]

> N2 is shaping up to be TSMC's most important node since N7

Is it?

N2, from an energy & perf improvement seems on par with any generation node update.

          N2:N3   N3:N5  N5:N7
  Power   ~30%    ~30%    ~30%
  Perf    ~15%    ~15%    ~15%

https://www.tomshardware.com/news/tsmc-reveals-2nm-fabricati...

[aurareturn]

Yes. It has more tape outs at this stage of development than both N5 or N3. It’s wildly popular for chip designers it seems.

[alberth]

I thought Apple gets exclusive access to the latest node for the first 1-2 years. Is that not the case?

[aurareturn]

No. That's not the case. Maybe for a few months only.

[alberth]

Correct me if I'm wrong but didn't TSMC launch N3 in 2022, and still only Apple uses this latest/smallest node.

Both AMD and NVIDIA are using N4.

[pseudosavant]

I love that term "refining energy". We need to plan for massive growth in electricity production to have the supply to refine.

[tmalsburg2]

Sounds smart but it’s abusing the semantics of “refine” and is therefore ultimately vacuous.

[pseudosavant]

I think it is really just the difference between chemically refining something and electrically refining something.

Raw AC comes in, then gets stepped down, filtered, converted into DC rails, gated, timed, and pulsed. That’s already an industrial refinement process. The "crude" incoming power is shaped into the precise, stable forms that CPUs, GPUs, RAM, storage, and networking can actually use.

Then those stable voltages get flipped billions of times per second into ordered states, which become instructions, models, inferences, and other high-value "product."

It sure seems like series of processes for refining something.

[jacquesm]

It is the opposite of refining energy. Electrical energy is steak, what leaves the datacenter is heat, the lowest form of energy that we might still have a use for in that concentration (but most likely we are just dumping it in the atmosphere).

Refining is taking a lower quality energy source and turning it into a higher quality one.

What you could argue is that it adds value to bits. But the bits themselves, their state is what matters, not the energy that transports them.

[elbasti]

I think you're pushing the metaphor a bit far, but the parallel was to something like ore.

A power plant "mines" electron, which the data center then refines into words. or whatever. The point is that energy is the raw material that flows into data centers.

[fuzzfactor]

Maybe more like converting energy to data, as a more specific type of refinement.

[phkahler]

Using energy to decrease the entropy of data. Or to organize and structure data.

[fuzzfactor]

I like that. Take random wild electrons and put them neatly into rows & columns where they can sit a spell.

[LaGrange]

This is OpenAI, they are not decreasing the entropy. This is refining coal into waste heat and CO2.

[reubenmorais]

All life is basically refining energy - standing up to entropy and temporarily winning the fight.

[HPsquared]

It's all about putting the entropy somewhere else and keeping your own little area organised.

[xnickb]

People of the earth, remember: unnecessary arm and leg movements increase the entropy! Fear of the heat death of the universe! Lie down when possible!

[antihipocrat]

Yes, in a very local context it appears so, but net entropy across the system from life's activities is increased

[ithkuil]

"the purpose of life is to hydrogenate carbon dioxide"

-- Michael Russel

[casey2]

Where do the cards go after 5 years? I don't see a large surplus of mid sized cloud providers coming to buy them (cause AI isn't profitable), Maybe other countries (possibly illegally)? Flood the consumer market with cards they can't use? TSMCs' more than doubled packaging and they are planning on doubling again

[protocolture]

This.

A local to me ~40W datacenter used to be in really high demand, and despite having excess rack space, had no excess power. It was crazy.

[nixass]

40W - is that ant datacenter? :)

[protocolture]

Yeah, it was the companies pilot site, and everything about it is tiny.

But it very quickly became the best place in town for carrier interconnection. So every carrier wanted in.

Even when bigger local DC's went in, a lot of what they were doing was just landing virtual cross connects to the tiny one, because thats where everyone was.

[inemesitaffia]

You lost a M or K next to your W.

I still have an Edison bulb that consumes more power.

[protocolture]

Yep I see that haha.

[pabs3]

> the power requirements are basically locked-in

Why is that? To do with the incoming power feed or something else?

[brendoelfrendo]

Basically, yes. When you stand up something that big, you need to work with the local utilities to ensure they have the capacity for what you're doing. While you can ask for more power later on, if the utilities can't supply it or the grid can't transport it, you're SOL.

[pabs3]

You could in theory supplement it with rooftop solar and batteries, especially if you can get customers who can curtail their energy use easily. Datacentres have a lot of roof space, they could at least reduce their daytime energy costs a bit. I wonder why you don't see many doing solar, do the economics not work out yet?

[brendoelfrendo]

I'd have to do the math, but I doubt that makes sense given the amount of power these things are drawing. I've heard of DCs having on-site power generation, but it's usually in the form of diesel generators used for supplemental or emergency power. In one weird case, I heard about a DC that used on-site diesel as primary power and used the grid as backup.

[XorNot]

Compared to their volume they absolutely do not: you get about ~1kW / m^2 of solar. Some quick googling suggests a typical DC workload would be about 50 kW / m^2, rising too 100 for AI workloads.

[jl6]

Cooling too. A datacenter that takes 200MW in has to dissipate 200MW of heat to somewhere.

[djtriptych]

guessing massive capital outlays and maybe irreversible site selection/preparation concerns.

[kulahan]

That's pretty interesting. Is it just because the power channels are the most fundamental aspect of the building? I'm sorta surprised you can't rip out old cables and drop in new ones, or something to that effect, but I also know NOTHING about electricity.

[libraryofbabel]

Not an expert, but it’s probably related to cooling. Every joule of that electricity that goes in must also leave the datacenter as heat. And the whole design of a datacenter is centered around cooling requirements.

[vrighter]

Exactly. To add to that, I'd like to point out that when this person says every joule, he is not exaggerating (only a teeny tiny bit). The actual computation itself barely uses any energy at all.

[pjc50]

Refining it into what? Stock prices?