[gzer0]

This is a great build, thanks for sharing your learnings.

The best build I have seen so far had 6x4090's. Video: https://www.youtube.com/watch?v=C548PLVwjHA

  Specifications
  - GPU Accelerator - 6 x 24GB NVIDIA GeForce RTX 4090
  - Processor - Intel Xeon W7-3465X, 28C/56T, 2.5GHz - 4.8GHz
  - Memory - 256GB (8x32GB) DDR5 ECC 4800MHz
  - System Drive  - 2TB Samsung 980 PRO NVMe PCIe 4.0 M.2 SSD
  - Storage Drive - 4TB Samsung 870 EVO SSD
  - Operating System - Ubuntu 20.04

An interesting choice to go with 256GB of DDR5 ECC; if spending so much on the 6x4090's, might as well try to hit 1 TB of RAM as well.

The cost of this... not even sure. Astronomical.

[manmal]

On Reddit there's reports of 8x4090, or even 8xH100. I don't know where people get this kind of money for this, and why they don't rent infra instead.

[FridgeSeal]

Probably because they are after a lot of fast, local storage, and _that_ is where rented ML infra providers will sting you.

Edit: could also just be more-money-than-sense. Never discount stupidity.

[dogma1138]

Hardware can be resold, and bought 2nd hand also.

The 4090 will likely maintain 50% of its current value due to its memory capacity over the next 12-18 months.

CapEx vs OpEx is a thing even if you are not a business…

[KuriousCat]

Why do you think RoI is better when infra is rented?

[taskforcegemini]

I mean, at some point someone has to buy them to be able to offer services on them to others. Renting comes with certain limitations owners don't have. And some people have too much money to not invest in fun.

[belter]

Don't forget to talk to your local power company one year in advance. They will need to upgrade your local substation transformer... :-)

[amluto]

This build is 3kVA max. That’s about 1/3 of a current gen EV, only 15% of an original Tesla Model S with dual chargers, and about equal to a standard American oven. This is much more polite to the grid than, say, a couple of tea kettles or especially a reasonably sized electric tankless water heater.