|
> A node of 8 H100s will run you $31.40/hr on AWS, so for all 96 you're looking at $376.80/hr And what stinks is that you can't even build a Dell/HPE server like this online. You have to 'request a quote' for an 'AI Server' Going through SuperMicro, you're looking at about $60k for the server, plus 8 GPU's at $25,000 each, so you're close to $300,000 for an 8 GPU node. Now, that doesn't include networking, storage, racks, electricity, cooling, someone to set that all up for you, $1,000 DAC cables, NVIDIA middleware, downtime as the H100's are the flakiest pieces of junk ever and will need to be replaced every so often... Setting up a 96 H100 cluster (12 of those puppies) in this case is probably going to cost you $4-5 million. But it should cost less than AWS after a year and a half. |