[kingstnap]

A DGX B200 costs like ~$0.5 M and uses around 14 kW.

If you plan to run it straight for 8 years 100% max usage thats around 1 GWhr.

A gigawatt hour is a lot of energy but its not that much compared to the price of the actual machine. In Germany for example with its expensive energy thats about €100k worth, which spread over 8 years is pretty minor compared to the up front half mill.

The real issue with high power consumption is not really the cost of energy but the limited powersupply you can get for a datacenter. A more efficient setup is highly desirable because it means you can fit more in the limited power hookup.

[bayindirh]

> but the limited powersupply you can get for a datacenter.

Since many people haven't seen 10MW cabling for a data center or how a big GPU server is cabled, they naturally imagine connecting servers is akin to plugging an appliance to a wall.

When the electricity provider says "I neither have the capacity, nor the required cables in that area", thing gets real.

[dannyw]

It’s more than power supply. Cooling and ventilation becomes a MUCH bigger deal at rack scale, and that costs electricity too.

[thereisnospork]

Cooling demand is only fractional with respect to the load: cooling 1MW of heat will only cost a few 10's to low 100's of kW, depending on the specifics. 10-20% overhead on cooling is probably a close enough estimate for napkin math.

[bayindirh]

With liquid cooling technologies (direct or rear door heat exchange), cooling efficiently is easier when compared to a decade ago, and it's pretty efficient when you compare the power consumption numbers (server total vs. cooling total).

See PUE (Power Usage Effectiveness) for its scientific form.

[psychoslave]

And datacenters have impact on everything around them. If at the end of the day to result is a few more yachts and jets and, a lot more of miserable humans starving in ruined ecosystems, maybe that’s not the best go-to direction.

[butvacuum]

You say they have a large impact, but having lived somewhere with some of the largest data centers- they very much don't. At least not more so then any other structure that paves over greenery.

love to debate actual discission points. pull up "datacenter dfw" on google maps for mine.

[well_ackshually]

a constant low 60dB 20Hz hum in the background, 24/7 is as close as a a torture technique invented by the CIA as it can get.

[ffsm8]

The people having glass literally break from the vibrations would probably disagree with your opinion

https://youtu.be/_bP80DEAbuo?is=sg09k66iutKFIFSo

Yet here we are, discussing "data center" as if they're standardized and of similar (nose) isolation.

There are no meaningful regulations in building them, and they can be incredibly polluting. So your experience with a potentially well isolated one is sadly not the norm going forward. And we don't even know how close you lived, if you're eg talking about "within 5km/3miles" then your experience would also have little value in this discussion in general.

[jml7c5]

>The people having glass literally break from the vibrations would probably disagree with your opinion

Can you cite a source for this? It's not in the video, as far as I can tell.

I would be wary of Benn Jordan's videos. They are full of mistakes and misrepresentations, as Andy Masley has convincingly demonstrated: https://blog.andymasley.com/p/contra-benn-jordan-data-center...

I recall seeing Benn Jordan's responses on Bluesky and thinking they were quite poor. He was unwilling to admit to mistakes, and kept trying to grasp at newly searched papers that didn't actually support his arguments.

[butvacuum]

It's irrelevant. Glass is a fickle mistress- a factory defect from 20 years ago can be the nucleus for the whole pane to shatter for any reason, or no reason.

Anything loud enough to crack any pane of glass that gets near it means federal agencies are very interested.

[hypfer]

Benn unfortunately is one of those people that actually feel stuff, which is a trait that easily gets exploited by bad actors.

Indeed, he shot himself in the foot there pretty bad, but I would argue that that was just the result of successful Agitation.

I would personally strongly prefer being in the same room with Benn compared with Andy, because one of them is authentic, while the other is calculating. Though, arguably, Benn has been catching up on that lately too.

But yeah, taking stuff with a grain of salt should be the default regardless of the person speaking.

[apublicfrog]

The fact that people have lived and worked near data centres for decades and didn't even know what the term meant - let alone be adversely impacted by them - probably indicates they're broadly an non issue. All of a sudden out of nowhere, AI and data centres got intermingled by the media and now people seem to have big issues with them.

[bayindirh]

Because the dynamics have shifted enormously inside the rack.

10 years ago, I was running 4 CPU servers with 48 cores and 128GB of RAM in 2U enclosures with a maximum power consumption of 500W or so. I was able to stick ~20 of them in a 42U rack, totaling 10kW.

A data center full of these can be cooled with CRACs and hot/cold aisles without much problem. This is still too much for a bog-standard server colocation operation, but for HPC, that was normal and manageable.

Now, a ~1U server houses 4 SOTA NVIDIA GPUs, 64 cores, magnitudes more RAM. This server alone uses ~3KW of power. This means you go anywhere between 30kW to 50kW per rack, and you have many racks.

Of course this means more power comes in, more heat comes out. This means more sophisticated infrastructure: bigger and beefier primary and secondary power systems, beefier cooling, more heat, more noise, in short "more of everything".

Of course when you cram this much energy and heat into a relatively small space, its effect on the environment will be much more pronounced.

Facebook's previous SOTA datacenter used water infused, HEPA filtered free flowing air accross the datacenter. Now, it's server level direct liquid cooling with extensive water treatment and oversight on coolant parameters.

Compare this having a hand warmer vs. coal ember in your hand. The latter needs a much more elaborate setup to prevent it burning you badly.

[Liftyee]

Though, the new data centers are not entirely the same. Increasing use of onsite gas turbines to generate power instead of using grid power changes their noise+air pollution profile.

[lnsru]

Sounds exactly like the stories with 5G cell towers. Almost no problems with GSM and then suddenly 5G is big issue.

[ninjalanternshk]

> There are no meaningful regulations in building them

If a municipality doesn’t have emissions, noise, water use, etc regulations, that’s a serious failure in governance.

We don’t need nor want the word “data center” in regulations anymore than we need the word “abattoir.”

The names of the things we build change all the time. Their impact on their communities don’t.

We need to regulate impact, not the name or type of business.

If we did, nobody would know or care about data centers and they wouldn’t be affecting their communities, because they’d be operating under established impact regulations.

[rvba]

Why are you using internet then? The (highly optimized) HN server sits in a rack too.

So you want to use the internet but dont want the data center in your backyard?

[rpdillon]

How far do you live from a data center?

[minraws]

It's not even about the costs, getting enough power for a large datacenter is impractically hard in most of the world at a single location.

If it's efficient and the power costs of not just ongoing costs but the upfront setup is lower that makes a lot different scales of data centers practical, especially for inference which doesn't need massive super clusters.

You can't just fire up gas turbines everywhere like US Data centers are doing. I am not even sure if that's legal in US...

Note you have to plan for peak usage and a lot of stuff large scale data centers are insane infrastructure projects.

Nvidia is both supply and price constrainted, sure if you are willing to pay over 0.5M$ you might get some, but if you try to balance out price to costs by going slightly lower on the pole you realize just how much more expensive Nvidia truly feels like AMD has a lot of margin to under cut them if they want to.

[willis936]

What they're really asking the authors is "can you not lie about performance cost and do proper accounting?". You can spin any story if you cherry pick your framing sufficiently. Stopping right at the silicon packaging boundary is as meaningless as it seems.

The article is highly qualified but the headline is not. If they are not making general statements then they shouldn't open with them.

[jwpapi]

Interesting so it’s supply chain and then you need to calculate how long it can be utilized and for how much you can sell it.

Would love more calculations on that

[heisenbit]

Plus the power needed for cooling adding maybe 50%.