[akreal]

I was curious about energy efficiency and took two samples from the linked MLPerf GPT3 results. H100 seems about three times more efficient than Gaudi2.

  256 Gaudi2 600W TDP: 256 * (442.578 / 60) * 0.6 = ~1133 kWh
  512 H100 700W TDP: 512 * (64.264 / 60) * 0.7 = ~384 kWh

[sm_1024]

I'm guessing H100 has 2x host energy overhead for connecting those GPUs? That might offset some of the perf/W benefits of nvidia's offering.