[Jabbles]

> Given the choice between having a team of 3 people working under me or 1000 dedicated H100s for my work, I’d have to think for a little bit.

But the exact numbers are important. An H100 is ~$2/hour [1], so 1000 is $16M/year (24/7). Even if Google gets a massive internal discount that's still way more than 3 people's total cost. If you have to choose between 16 (highly paid, senior) people or 1000 H100s would you have to think about the choice?

Then again when you revisit this comment in a few years' time the original comparison may be correct.

[1] https://gpus.llm-utils.org/h100-gpu-cloud-availability-and-p...

[ExoticPearTree]

I don’t think the author meant for this to be treated as an absolute, more like food for thought.

And if you really insist on 24/7 comparison, you would come to need about 12 people, as the expected productivity per person is of about 6 hrs/day. Factor in the fact that people need vacations, sick days, weekends off… it looks like 1000 H100 might actually be a good trade-off in the very near future.

[silvestrov]

purchase price != sales price.

Very unlikely that Google pays $16M for a single H100. Amazon has it at $44k: https://www.amazon.com/Tesla-NVIDIA-Learning-Compute-Graphic...

[inexcf]

Better reread the post. It was about renting GPUs. 16M for 1000 H100 per year. So i guess 44k at Amazon would make that 44M Dollar.