[Dylan16807]

Depends on what you're doing. I'm pretty sure the bandwidth for inference isn't much.

[eurekin]

Depends, if it's tensor parallel or pipeline parallel. Only PP doesn't pass too much. TP does