[sireat]

I was under the mistaken impression that you could not go beyond 2x3090 for reasonable inference speed.

My assumption was that going beyond 2 cards incurs significant bandwidth penalty when going from NVLink between 2x3090s to PCIe for communicating between the other 3090s.

What kind of T/s speeds are you getting with this type of 8x3090 setup?

Presumably then even crazier 16x4090 would be an option for someone with enough PCIe slots/risers/extenders.