[twtw]

Great work, RiseML. This benchmark is sincerely appreciated.

I wonder whether NVLink would make any difference for Resnet-50. Does anyone know whether these implementations require any inter-GPU communication?

[elmarhaussmann]

They don't require it but some of the ResNet-50 implementations can make use of it (e.g., the ones in the Docker containers on the Nvidia GPU Cloud). But even the ones without seem to scale to 4 GPUs pretty well. This may be a different story for 8 GPUs and larger/deeper networks, e.g., ResNet-152.