[jfkfif]

the problem is multinode runs that communicate through the network

[freeone3000]

Multinode runs don’t communicate through the network in a DGX configuration. NVlink allows for RDMA over direct infiniband. No need for network here.

[tomoyoirl]

Infiniband is a network too…

But even if we set that aside you’ll get access to your data over a network connection because these are expensive nodes running batch jobs with finite disk space, not personal workstations.

[freeone3000]

Yea ofc. Nvidia has for mellanox infiniband, nvlink external, and even mellanox ethernet pci-to-pci, no need to involve the CPU. nvidia-docker has a few mods to support this too.

[josh-sematic]

Yes, which is especially important for training. Getting good GPU interconnect can be really important for training large models.