[vomitcuddle]

Does Nvidia already have NVLink for this purpose?

[jsheard]

They do, but you can't run NVLink to the CPU unless you're using exotic IBM POWER8 systems.

x86 systems like the DGX-1 can only run NVLink between the GPUs, and still rely on PCI-E for the CPU connection.