[patrick-fitz]

Out of curiosity, looking at the cheapest price for a H100 that I could find online.

Lambda Reserved Cloud [1] starts at $1.89 per H100 per hour.

It could be possible to get the cost down to a lower amount:

$1.89 * 96GPUs * 24hours * 14days = ~$61k

1 - https://lambdalabs.com/deep-learning/servers/hyperplane

[avrionov]

This is the price of training if nothing fails.

[logicchains]

It also depends on the interconnect speed. If you don't have fast enough interconnect between the machines, you won't get linear speedup with N gpus.