[everforward]

I'm doubtful they can, based on current inference prices. An H100 draws about 50W while idling, which is ~$8/month at average US electricity prices. They also draw ~400W while active.

The electricity prices are relevant because if you paid $0 for your H100 and didn't use it a single time, you could buy millions of tokens in inference just on the electricity it draws while idling. If you can't keep that thing saturated through the night, you're probably underwater overnight. Likewise, it's too small to run even the frontier open source models so you need to be able to live with worse models.

Max power matters because you aren't going to run many of those H100s before you blow breakers in most houses. Newer houses in the US are 15A service to non-kitchen breakers, so 1650W (that might be peak rather than continuous, not sure). If you're plugging that into an existing run, you could maybe run 2 before you start blowing breakers? You can't just plug 4 H100s into the wall in a normal house.

Maybe I'm wrong, though. I'd be curious, it'd be neat to run my own inference for something more than what'll run on a 3080.

[smalltorch]

Most houses have a 200amp panel and extra space for larger breakers.

A electrician can plop in a electric car charger for instance, that is a 40-50 amp circuit.

[everforward]

The median home age ranges from 26 to 63 depending on state, and 100 amp service was common into the 90s. 100A service is probably more common than you think, I'd bet we teeter around them being equally common.

My home still has 100 amp service (as does almost the whole city, most of it is a historic district), but it was built in 1910 so that's not shocking.

> A electrician can plop in a electric car charger for instance, that is a 40-50 amp circuit.

Oh yeah, I was mostly talking about having to upgrade the service to the house from the pole. A new circuit with a single outlet isn't horribly expensive, but needing to upgrade service is pretty rough.

[rtkwe]

Newer houses yes probably have the overhead. We put in a charger for my wife (because she drives 100 miles a day we needed a pretty beefy level 2) and had to replace our whole panel because 1) we had 125A so failed the load calc and 2) we had the infamous Zinsco panels and the electricians refused to squeeze one in even if we did pass the load calculations. Luckily we had a basement so they only had to extend the circuits a little instead of rerunning them all over the house.

[HWR_14]

I'd imagine that you can have the power to the board shut off when not in use.

But I never assumed it was to be cost competitive at current token rates. I assumed it was for the same reasons people might use open source hardware. Freedom to tinker, etc.

[rtkwe]

Basically sounds like a hobo version of PCI-e hotswapping which is a thing on enterprise motherboards but not many consumer boards. Also depends on if the board can actually power off the slots which I'm not sure many are setup to do.

[trollbridge]

A typical oven plug can offer 9.6kW (so your 24-node H100 can keep your kitchen nice and warm). My furnace was 16.5kW, which once failed in an odd way and was stuck on for about a week. The people living in the house simply opened the windows until I got around to fixing it. I estimate it wasted $400 on the power bill.