[modeless]

I wonder how the cost compares to a Tinybox. $25k for 6x 4090 or $15k for 6x 7900XTX. Of course that's the full package with power supplies, CPU, storage, cooling, assembly, shipping, etc. And a tested, known good hardware/software configuration which is crucial with this kind of thing.

[Tepix]

If you merely want CUDA and lots of VRAM there‘s no reason to pick expensive 4090s over used 3090s

[halJordan]

Well there is and it's called performance. You dont have to push your version of what an appropriate price/performance ratio is

[Tepix]

Well, if you run out of VRAM, you drop off a performance cliff. That's a whole order of magnitude slower than just using a slower GPU but fitting everything into VRAM.

[angoragoats]

You can build a setup like in the OP for somewhere around $10k, depending on several factors, the most important of which are the price you source your GPUs at ($700 per 3090 is a reasonable going rate) and what CPU you choose (high core count, high frequency Epyc CPUs will cost more).

[itomato]

With a rental option coming, it’s hard for me to imagine a more profitable way to use a node like that.