[WithinReason]

That's 90% bandwidth efficiency and 60% compute efficiency

https://www.nvidia.com/en-us/data-center/h100/

[helloericsf]

They don't have h100. wink,wink.

[rfoo]

They have H800s which have exactly same memory bandwidth and max FLOPS.

[pk-protect-ai]

What about NVLink? Does it plays a role here?

[rfoo]

For FlashMLA? No. The code here runs on one GPU only and do not have a builtin communication part.

[pk-protect-ai]

But for the training it does. You need to communicate gradient changes between GPUs.