[DiabloD3]

Whats wrong with Infiniband? Cost is typically less than dealing with Ethernet NICs that can do RoCE.

Intel is also moving ahead with their Infiniband implementation as the "supercomputing" cross-chassis bus (QPI is a clone of Hypertransport, but Intel decided not to use it for supercomputing bus fabrics like AMD does with their externally switched Hypertransport implementation), so future Xeons and future Phis will have on-die True Scale controllers.

[Keyframe]

Nothing's wrong with infiniband, of course! Except for prices compared to average SoHo network infrastructure. Video mastering and editing networks usually aren't large. You need a fast SAN with fast network and up you go. Everything around it is so expensive though. In recent years there have been a massive price drop in fast storage, fast graphics cards and now we wait for cheap fast networks and 10-bit through and through.

[DiabloD3]

Exactly how fast are we talking? Dual 10gbit nics are now often for $150 + cost of optical modules or DACs.

[Keyframe]

We're talking mastering here. For editing it doesn't matter since it can and is done in proxies. Depending on the bit-depth and fps we're talking anywhere between ~127 MBps to ~253MBps (1+ - 2+ Gbps) per workstation. Usually there are several and a one heavy NAS or several smaller ones. That's for 2K (and HD - but then it's a bit smaller bandwidth). For 4K it's four times that.

[scurvy]

Cabling I'd suspect. The copper cables are thick, the fiber QSFPs are expensive.

[DiabloD3]

The fiber QSFP modules are the same ones used for Ethernet, ergo, 40gbit Ethernet and 56gbit Infiniband are going to have the same issue with QSFP modules being expensive.