[justincormack]

the reason Nvidia only talk about two machines over the network is I think they only have one network port, so you need to add costs for a switch.

[Cane_P]

It clearly have two ports. Just watch on the right side of the picture:

https://www.storagereview.com/wp-content/uploads/2025/01/Sto...

You will however get half of the bandwidth and a lot more latency if you have to go through multiple systems.

[bick_nyers]

If you want to split tensorwise yes. Layerwise splits could go over Ethernet.

I would be interested to see how feasible hybrid approaches would be, e.g. connect each pair up directly via ConnectX and then connect the sets together via Ethernet.