[tyfon]

I half expect Nvidia to have buyback contacts like Ferrari with the larger customers to prevent a price crash when they all upgrade and to keep them scarce.

I hope not though, perhaps I can pick up a H100 in a few years if they get sold on the open market.

[rickypp]

I worked at an org that had a substantial on-prem GPU datacenter. We transitioned to <Big Cloud Provider> with a substantial negotiated discount rate, with part of the contract being we would sell them all of our hardware and not purchase any more.

[hamandcheese]

> and not purchase any more.

why would anyone sign such a contract?

[toast0]

If you're planning to run in clouds, committing to not buy hardware (during the contract term, presumably) isn't a big imposition. Maybe you switch to a different cloud, and you wouldn't buy hardware for that.

If you want to switch back to on prem, there's probably a way to structure acquiring hardware so it doesn't break the contract. Maybe you lease it, maybe the purchase happens through a related company, maybe there was no way for the contracted cloud to find out...

[collabs]

I don't know if this was IT shrugging me off or if it was something real but some IT person at this big ISP I worked at told me that they cannot just buy an SSD — my windows box at work was running off of a hard disk in 2019 — and that there was some contract that said any computer hardware we bought had to be through HP or something like that and it takes many months it something like that.

[lelanthran]

> I don't know if this was IT shrugging me off or if it was something real but some IT person at this big ISP I worked at told me that they cannot just buy an SSD — my windows box at work was running off of a hard disk in 2019 — and that there was some contract that said any computer hardware we bought had to be through HP or something like that and it takes many months it something like that.

Plausible - the hardware might be leased, and so you have no right to modify it. You have to pay them to modify it.

Same as if you leased a car, you cannot do the services yourself, you have to pay them (and an approved agent of theirs) to do it.

[thijson]

We could only book our business travel through American Express. It must have made economic sense at the c suite level, I wasn't seeing it at my level, the flights were consistently more expensive.

[protocolture]

Was it a HP Finance thing? Because there are a shit ton of people who can transact through them now. We had HP finance, and it became a game of sorts to see what crazy shit we could get with it. Theres a local mob who sell computer parts who were happy to use HP finance. That said it definitely wasnt everyone and if you didnt do the legwork you could definitely be trapped.

[Shorel]

The usual very short term corporate thinking that maximizes quarter profits while bankrupting the company in the long term. IMO a very shortsighted decision, if not downright stupid.

[vasco]

What is short sighted is parroting this over and over. It's not insightful, interesting or adding anything else to the discussion than "mean people are bad".

[brookst]

It’s also possible to bankrupt a company with excessive focus on the long term.

[Shorel]

It would be very hard to find current examples of companies doing that harmful excessive focus on the long term. Only Valve comes to mind and they are anything but bankrupt. Quarter profits maximization will create those loops of doing and undoing the same thing. If you check the successful companies (Apple, etc.) their activities are mostly year cycles.

[fsuts]

Because the discount would be sufficiently large.

[lithos]

Short term thoughtset MBAs.

[throwaway85825]

How does this work? Is the OEM giving the cloud provider a big discount or is the cloud provider giving you a teaser rate to lock up your business.

[wmf]

The list price of cloud is 10x higher than the price of hardware itself so that leaves room for some discounting.

[jurgenburgen]

What I’ve learned is that the list price is just for small companies. When you’re big enough you ask for a better contract.

[to11mtm]

Yeah....

Where folks (often) get lazy is the resulting math over what the real bean-counters care about (but are too lazy to check often).

In a past life, I worked on costing models for a Cable/Fiber contract house, to help the company decide 'what was profitable to keep in house' versus 'what do we subcontract' (sometimes that could even mean we just 'rented' a machine and had a qualified operator using it, based on that employee's hourly rate and expected L2R for taxes... so many spreadsheets...)

And from from my 'I don't know all the factors for this but I've seen how people screw up the big ones' view (and frankly, I'm guessing a lot of us have seen and dealt with the same category of 'bad math' around outsourcing IT work...)

An on-prem data center means:

- You need to account for electricity costs - i.e. CA vs midwest electric rates.

- cooling and power backup capability - Smaller factor but real

- personnel cost - e.x. there's probably cases where a smaller org could be better off with 'on-site' server admins that have other roles based on local wages. Kinda case specfic but it's a case.

- whatever the 'space' holding the stuff costs

  - Sardonic take :Hey, let's have another unused meeting room instead! (e.x. In the case of on-prem shops that simply fled to AWS in their migration from VMware)


 - the cost of licensing whatever is running

  - In defense of this, In one of my earliest IT lives, AWS handling the Oracle licensing for a DB was a *huge* win as far as making it as easy as possible to ensure whatever was going on we couldn't have the Oracle licensing folks 'ding' us on whatever infraction occurred between reviews (that could not be understood by the majority of the company, often including the accused. I was never guilty but I saw it happen to others.)

  - OTOH I know lots of folks who just want to be lazy about what they have to document.

Still, IMO a lot of orgs don't do the right math around these decisions, or just buy into the 'Well trends can change' as though they can decide as an org they need to suddenly triple capacity in a month and it would be able to organically happen in the first place.

Frankly, the orgs that 'might' need that either have their arch set up where they are in cloud, or they are onprem but can scale to cloud if needed in interim.

[lmm]

What are these orgs that have unused meeting rooms? Everywhere I've ever worked has had a chronic meeting room shortage.

[moffkalast]

The only reason a datacenter would ditch their H100 is if it becomes uneconomical to run them, with newer silicon providing much more power efficiency. When that happens they'll look like a used V100 looks today: horribly inefficient, lacking modern data types and engines, requiring screaming server fans with weird adapters to not melt, way beyond end of life in terms of cuda support. Almost completely damn useless unless you really have no other alternative.

[imtringued]

Honestly, the primary reason you would ditch a H100 is because the ML models have moved onto a different data type, e.g. nobody uses fp8 anymore and everyone jumped onto fp4 or some custom block float thing.

[nylonstrung]

There's going to be a golden age of GPGPU compute in the next few years once A100/H100 are fully obsolete for running frontier models efficiently and the price plummets

It will be perfect for stuff like GPU-accelerated query engines, "classical ML" and every other CPU-based workload that could conceivably be offloaded to GPU

[nl]

There's nothing stopping you doing this now.

You can get used 16GB P100s on AliExpress for ~$100 if you want obsolete GPUs. Allegedly new AMD BC 250s are only slightly more.

I've looked at this some but I already have a GTX1070 which is only supported upto CUDA 11.9.

That's precludes some interesting modern optimizations out of the box. I've spend a lot of LLM tokens backporting some things, but I'm really not sure the hassle is worth it.

New hardware is just better. I think in maybe 5 years when supply and demand are back in equilibrium we are going to have some killer technology for decent prices, and 15yo H100s won't look attractive.

[throw0101d]

> You can get used 16GB P100s on AliExpress for ~$100 if you want obsolete GPUs. Allegedly new AMD BC 250s are only slightly more.

In a recent Gamer Nexus video with Level 1 Tech, they mention V100s are also quite useful for many applications that use FP64: stuff four in a workstation, and many PhD candidates would be quite happy with the throughput they can get for certain scenarios.

[Schlagbohrer]

Linux hackers will be finding all kinds of crazy uses for hardware that now costs $100,000 and in 5 years will be available as scrap.

This is assuming that there is no big, big disruption to the semiconductor industry (e.g. TSMC getting attacked), in which case... well, I am gonna treat each stick of RAM I currently have like it's a faberge egg.

[throwaway894345]

GPGPU? General Purpose GPU? If embarrassingly parallel CPU algorithms weren't offloaded to the GPU previously, why would the A100/H100 price drop make a difference? We had cheap GPU in the past and we still left plenty of performance on the table with CPU programs because they were easier to build.

Is the idea that previously maintaining GPU programs was expensive whereas now AI makes it cheap? If so, I could buy that line of reasoning.

Maybe relatedly, I expect (hope) the hardware manufacturers will ramp up supply in the meanwhile which would also put downward pressure on GPUs. Right now though this hardware crunch is making me sad, not even because of GPUs but also because of general memory / disk.

[christina97]

GPGPU programming has become significantly easier now and the payoff is bigger (better hardware), due to the immense investment in this due to ML/AI.

[NuclearPM]

What are the best tools for this?

[girvo]

We've not had cheap GPUs with this much VRAM before, though. Might be an interesting change, though I also doubt it personally.

[nl]

As noted in my other comment you can get obsolete GPUs (P100s, BC250s) with lots of RAM on AliExpress now. It hasn't proven revolutionary.

[swiftcoder]

is 16Gb "lots of RAM" in an LLM world? How many of these would you need to stack on a motherboard to inference a decent size model?

[oblio]

At least 4, probably 6+.

[fc417fc802]

BC250 shares memory with system. 16 GB GPUs have long been available to consumers for a small premium. What's never been readily available before is 40+ GB of HBM.

[girvo]

Those don't have a lot of RAM though. Not like these 40, 80GB ones.

[adityaathalye]

I am rooting for literally this... Blogged here: After "AI": Anticipating a post-LLM science & technology revolution https://www.evalapply.org/posts/after-ai/

TL;DR.

> I, for one, welcome the coming age of the post-LLM-datacenter-overinvestment-bust-fueled backyard GPU supercomputer revolution.

> The Big Question is…

> Who is cultivating the option to snap up and repurpose vapourised datacenter investments at fire sale prices, soon as the "datacenter debt" cometh calling?

[echelon]

Wouldn't that be crazy - a hobbyist market for H100s?

Maybe someone could start a business buying up and rehousing these.

[rtkwe]

They're pretty specific to the datacenter use case with no outputs and they need to be cooled externally, principally through the very loud high speed fans used in data centers. I suppose you could strap a fan to one and put it in a normal case or maybe make a dedicated after market cooler (like the water blocks made for water cooling cases).

[jtolmar]

I think there's a market for a home AI server that can run an LLM or video gen model behind a web frontend. Not literally a raspberry pi strapped to an H100, but something with lopsided enough specs that people joke it is.

And precisely because it's such a huge headache to do yourself, I think a small company could make a nice business wrapping up used datacenter cards in that sort of server.

[everforward]

I'm doubtful they can, based on current inference prices. An H100 draws about 50W while idling, which is ~$8/month at average US electricity prices. They also draw ~400W while active.

The electricity prices are relevant because if you paid $0 for your H100 and didn't use it a single time, you could buy millions of tokens in inference just on the electricity it draws while idling. If you can't keep that thing saturated through the night, you're probably underwater overnight. Likewise, it's too small to run even the frontier open source models so you need to be able to live with worse models.

Max power matters because you aren't going to run many of those H100s before you blow breakers in most houses. Newer houses in the US are 15A service to non-kitchen breakers, so 1650W (that might be peak rather than continuous, not sure). If you're plugging that into an existing run, you could maybe run 2 before you start blowing breakers? You can't just plug 4 H100s into the wall in a normal house.

Maybe I'm wrong, though. I'd be curious, it'd be neat to run my own inference for something more than what'll run on a 3080.

[smalltorch]

Most houses have a 200amp panel and extra space for larger breakers.

A electrician can plop in a electric car charger for instance, that is a 40-50 amp circuit.

[everforward]

The median home age ranges from 26 to 63 depending on state, and 100 amp service was common into the 90s. 100A service is probably more common than you think, I'd bet we teeter around them being equally common.

My home still has 100 amp service (as does almost the whole city, most of it is a historic district), but it was built in 1910 so that's not shocking.

> A electrician can plop in a electric car charger for instance, that is a 40-50 amp circuit.

Oh yeah, I was mostly talking about having to upgrade the service to the house from the pole. A new circuit with a single outlet isn't horribly expensive, but needing to upgrade service is pretty rough.

[rtkwe]

Newer houses yes probably have the overhead. We put in a charger for my wife (because she drives 100 miles a day we needed a pretty beefy level 2) and had to replace our whole panel because 1) we had 125A so failed the load calc and 2) we had the infamous Zinsco panels and the electricians refused to squeeze one in even if we did pass the load calculations. Luckily we had a basement so they only had to extend the circuits a little instead of rerunning them all over the house.

[HWR_14]

I'd imagine that you can have the power to the board shut off when not in use.

But I never assumed it was to be cost competitive at current token rates. I assumed it was for the same reasons people might use open source hardware. Freedom to tinker, etc.

[rtkwe]

Basically sounds like a hobo version of PCI-e hotswapping which is a thing on enterprise motherboards but not many consumer boards. Also depends on if the board can actually power off the slots which I'm not sure many are setup to do.

[trollbridge]

A typical oven plug can offer 9.6kW (so your 24-node H100 can keep your kitchen nice and warm). My furnace was 16.5kW, which once failed in an odd way and was stuck on for about a week. The people living in the house simply opened the windows until I got around to fixing it. I estimate it wasted $400 on the power bill.

[Bombthecat]

Meh

Models are too big now

[timmmmmmay]

This is also true of the older P100 and hobbyists do this stuff now, today

[chasd00]

> Very loud high speed fans

if you haven't heard a 5u server intended for a datacenter rack come to life it's quite the experience. Sounds like a plane taking off.

[gerdesj]

"Sounds like a plane taking off."

Helicopter. I live in Yeovil, Somerset, UK - there's a helicopter factory just down the road. I had a IBM "AS/400" or whatever they are called now in our computer room rack for a customer and it made nearly as much noise as everything else put together. It was clearly tuned for start up noise to impress because they would fire up in sequence, rise to a crescendo and then slow down in sequence to just a din instead of painfully loud.

A switch or PC server on boot will normally run up cooling fans instantly to max as a default protection mechanism until the "OS" has started and sensors read and then the fans will slow down to deal with the actual thermal load.

If you switch off your air conn, it gets noisy, quickly. Recently in the UK we are seeing routine temperatures around 30C and we broke 200 odd year records for temperatures a few weeks back. I know its even worse elsewhere but our infrastructure is not designed for this. Here we are at the same latitude as Calgary AB!

[LargoLasskhyfv]

That's called staggered spin-up. One does that for sets of HDDs, too.

Anyway, what is it good for? To not overload the PSU for one, because spin-up needs more energy, until it has reached it's intended range of RPM. (for fans & HDDs)

Another reason to spin-up high initially, to fall down to slower when up, is to overcome friction in the bearings, to get stuck things unstuck, and shake dust off. (for fans)

[gerdesj]

Tell that to my switches and PC servers. They all spin up their fans in one go as soon as power good is received and the initial BIOS thingie has stubbed out its fag on the floor.

The PSUs are painting their nails when the fans go off initially. The GPUs, RAID controllers and co are still waking up.

A further counter example is an Equallogic PS6500. That has 48 3.5" spindles in it and three PSUs plus a few fans and two controllers. The whole thing boots in a couple of minutes.

[rbanffy]

My workstations are usually tower servers, which are the same design as their 4 and 5u rack counterparts.

Until the thermal management kicks in, they sound like jet planes. When the thermal management starts and assesses the required cooling, it’ll throttle down the fans to reasonable levels.

That is, until the moment you push the machine to its limits. When then happens, you might get back to the same levels of the boot time, but it’ll require you to push everything to the max - CPU, memory, storage (all 24 bays) and so on. For a normal user, there is a lot of room and it’s virtually impossible, even with a dozen of Teams windows open.

[baby_souffle]

Yes, but the line between "reasonably quiet" and 100% isn't a step function.

If these do end up in Home Labs, it's going to be a server rack in the basement and not the server rack at the other end of the room.

[rbanffy]

> If these do end up in Home Labs, it's going to be a server rack in the basement and not the server rack at the other end of the room.

If they are too loud, yes. In my part of the world basements are not common, but my shed would be a sensible place for one.

[CamperBob2]

Closest thing I can think of is the old LucasFilm "THX" trailer. Only there's no release segment, not until the BMC comes up a minute or so later.

[gessha]

If you check eBay for V100 SXM variant, you will often see them sold in combination with a cold plate, together with a PCIe carrier board. There’s definitely a market for them.

[lelanthran]

Meh.

You don't need much of a fan if you are prepared to pump a lot of water.

I mean, sure, if you're trying to cool the thing with 100ml of water, then yeah, you need a fan.

OTOH I have an unused car radiator in my garage, a quiet and cheap pump, and space to hang that radiator outside the window. It's barely an afternoon's worth of work if the H100s already have liquid-cooling intakes/exhausts on them already (I dunno, I have never seen one in the flesh).

I'm pretty certain 10+ litres of constantly circulating water (antifreeze, in my case) through a car radiator would be sufficient to cool down 2x H100s. Hell, add in 10 RPM used car cooling fan and I can probably cool 20x of those running at full-bore.

If anyone wants to ship me a bunch of H100s, I'll happily build it all out, take pics and videos and post it back here. Just sayin' :-)

[bayindirh]

Or you can secure yourself a DLC one and try to feed it with the correct regime (liquid composition, temperature and flow rate). I'd say good luck.

These things get hot and are fussy about their requirements.

[wmf]

H100s (and the PCIe converter card you need) are available on eBay.

[trollbridge]

Yeah, but an RTX 6000 is better for almost any application other than training, and the latter only really is better if you’ve got multiple H100s.

[mschuster91]

The GPU alone has a TDP of 700W, together with everything else (CPU, RAM, storage, fans) you're looking at 1500W+. Depending on the country, that may be enough to saturate your home's electricity uplink...

[holoduke]

Hell no. 1.5kw is half a socket capacity. A heat pump or many kitchen appliances are using much more. Our car charger uses more as well.

[jermaustin1]

That's why they said "depending on country"

In North America we are on 120V, making a standard 15A outlet only 1500W max, and something like 1200W sustained. To use higher wattage appliances, we have to upgrade our outlets to 20A (2000/1600W) or up our voltage to 240V, but that carries a different set of plugs and outlets as well.

[datadrivenangel]

Usually the home service is 200+ amps these days which is 24 KW total across all circuits, though you're right that most individual circuits are only 10-15 amp.

[Kirby64]

48kW across all circuits. The 200 amp rating is at 240V for US electrics.

You’re limited to 24kW per leg if you ran everything only on 120V, but any really large server is going to be running off 240V anyways.

[cyberax]

> In North America we are on 120V, making a standard 15A outlet only 1500W max, and something like 1200W sustained.

It's 1800W for short periods and 1500W sustained.

[trollbridge]

1,800 watt for periods less then 3 hours and 1,440 watts for continuous loads over 3 hours.

Realistically, a 15A breaker won’t trip on a 2,300 watt load for at least a few minutes, and often not until a few hours. A listed breaker is expected to trip in several seconds to 3 minutes for a 3,600 watt continuous load.

Take all those numbers and go up by 33% for a house or apartment with 20A circuits (quite common). Go up by 667% for an oven plug.

As anyone who takes care of rentals during winter knows, a typical circuit can tolerate two 1,500 W space heaters without tripping very much (although this is unsafe and a bad idea).

[jermaustin1]

Sorry, I was doing conservative napkin math, better safe than burnt to a crisp, as grandpa NEVER said... I'm currently having to rewire a lot of the house that he built and I bought so he could retire. There have been a lot of electrical issues that the inspection never found.

He hated outlets, and hardwired almost every appliance directly into whatever circuit was closest that fit the amperage, including literally cutting the plug off a cord, to nut the wires directly into the Romex.

I still haven't found what circuit my range hood is on. I turned off all 120V and it remained blowing, and knowing grandpa, he powered it off a single leg of a 240V. And I know he didn't use junction boxes, so the splices are likely all inside the walls.

[tyfon]

In the US they have a 120V system so it might actually saturate a normal socket over there. My PC room has a 16A fuse and 230V though, so should be plenty :)

[AshleyGrant]

The US has a 240V system for residential power, not a 120V system.

Most homes in the US have 200 amp service. That's 48 kW of power available to the house, though code states that you can only pull 80% of that continuously, so 160 amp/38.4 kW.

The 120V misconception comes from the fact that it is delivered as split phase on two 120V legs. Any competent electrician can run a 240V, 50-amp (40 amps/9.6 kW continuous) circuit to any room in a house. Plenty of homes have these circuits for electric ranges, EV chargers, or RV power outlets, and there's nothing stopping anyone from having one of these circuits installed in whatever room they'd like it installed in.

Heck, my old landlord and I installed one ourselves to provide an outlet for an EV charger in the garage of a house I lived in about a decade ago.

400 amp and even higher service is available, though it is fairly uncommon. Plenty of large houses will have a 400-amp service, though, which means you can double the power numbers available that I mentioned above.

[chollida1]

Most North American homes would have 240v and 200Amp service. A 16Amp fuse seems down right tiny:)

We have 50 amp service running a hot tub and another 50 amp service line to charge an electric car.

[tyfon]

The 16A is for the PC room, the house has a 40A "400V" 3 phase TN connetion :)

The quotes are in since the difference between the phases is 230V, but in practice it is almost the same as a 40Ax400V single phase connection.

[InsideOutSanta]

That's actually less bad than I assumed. High-end gaming GPUs are already almost at 600W with peak usage above that. I assumed it was much worse than that; that seems absolutely feasible for running at home.

[throw0101d]

> […] you're looking at 1500W+. Depending on the country, that may be enough to saturate your home's electricity uplink...

1500W is the power of the typical American microwave or (tea) kettle at 120V. (Convert over to a NEMA 6 plug and dual-pole breaker and you can get 3000W.)

3000W is nothing special in the rest of the world with >200V wall plugs.

[elictronic]

Never heard electrical uplink before but to those wondering Italy, India, and Japan all have requirements around 3kw. I was slightly surprised by this, but in the end if your running this in a tiny space with that low of power your already probably not buying used H100s.

[httpz]

A typical hair dryer uses about 1500W. I guess GPUs are mechanically not much different from a hair dryer.

[Crunchified]

Install it in your bathroom, use it like a restroom hand dryer.

[rbanffy]

> use it like a restroom hand dryer

One that runs continuously

[someguyiguess]

That should be illegal. Sounds like a very fraudulent business tactic.

[rcxdude]

Fraudulent, I wouldn't say so. Anti-consumer or anti-competitive? Sounds like.

[rbanffy]

It’s common for car companies when they enter a new market. It removes uncertainty from the second hand market.

By doing that, you know upfront what the value of your used hardware will be at the time you decommission it. It removes a lot of the risk for buyers in a volatile market.

[KaiserPro]

Its restrictive, but not that different from other types of deals.

For example, when I worked at a VFX software company, we were exclusively with one hardware partner. This unlocked something like a 50% discount across all our infra needs (they were big enough to provide switches, racks, servers, storage)

another company it unlocked a 75% discount.

[nl]

The hyperscalers signing these contracts have decent legal departments. Think about Oracle for example - I'm pretty sure they know every trick there is about beneficial contract drafting.

I don't think they need some special protection against this kind of contract.

[lelanthran]

> That should be illegal. Sounds like a very fraudulent business tactic.

a) It's not fraudulent, and

b) Nvidia has already signed on to buyback any unused capacity from the DCs it is selling to.

[throw1234567891]

Would you call a trade-in a fraudulent tactic?

[lmm]

Trade ins are fine. A contractual commitment to not buy hardware sounds like illegal restraint of trade.

[brookst]

How? It’s a contract. Should I be allowed to pay a landscaper less if I agree to not do part of the work myself?

I don’t understand the desire to tell totally anonymous third parties the terms under which they are allowed to deal with each other.

[lmm]

> It’s a contract

Many kinds of contracts are illegal, including those that are anticompetitive in certain ways.

> Should I be allowed to pay a landscaper less if I agree to not do part of the work myself?

No?

[InsideOutSanta]

Trade-ins are voluntary.

[efficax]

so are buybacks. you choose to sign the contract. there's no way they didn't have an escape clause, although likely it meant not using the cloud provider anymore

[InsideOutSanta]

> so are buybacks. you choose to sign the contract

Right, so they're not voluntary.

[anomaly_]

Literally what? Do you understand what is being proposed?

[zenbane]

it's also cruel. thankfully orgs like www.siliconstrays.com

[cyanydeez]

The Grift Economy places all legalities on the marks and their inability to form legal fights.

[sidewndr46]

isn't Ferrari the brand that requires any purchaser to be an existing owner? I could see NVIDIA going for something like that.

[metadat]

Invite-only is only for special edition hypercars and halo models. Standard models can be purchased by anyone with the funds and desire.

[m463]

tesla did that for the last 250 model s cars.

https://electrek.co/2026/04/11/tesla-signature-series-model-...

[driverdan]

Porsche is doing that for RS cars.

[olyjohn]

Yeah and now they have lost 98% of their profits in the last year. How's that catering to the ultra-wealthy working out for them? They used to build attainable cars that were nearly as cheap as a Corvette. Now they're double the price, and most certainly aren't double the performance.

[rgmerk]

From what I’ve read the only part of their business that is holding up is the 911 and particularly the high-end variants. It’s everything else that’s struggling.