[ryao]

Others can build infiniband hardware according to the standard. There used to be at least two companies building infiniband hardware until Intel killed QLogic’s infiniband division in a misguided attempt to make its own monopoly. :/

[mrlongroots]

Or they can develop a RoCEv2 that works (which is basically what UltraEthernet is).

Infiniband is not fun - it's a special snowflake of an interconnect that sits parallel to the rest of your datacenter network, and can not really run a standard TCP/IP codebase (yeah IPoIB is a thing but still). Do the Nvidia boxes really need a scale-out IP network as well as an Infiniband network?

Plus the spec is old. Packet spraying and trimming, better ordering guarantees, queue pair scalability... a whole bunch of enhancements have been incorporated into UE all the while being compatible with regular Ethernet.

Qlogic was never really an Infiniband vendor --- their qib driver is still in the Linux codebase and essentially emulates verbs on top of a messaging-based design.

[ryao]

We must agree to disagree then. Infiniband is an awesome technology. It was originally meant to to serve as a fabric to connect all components in a computer. That plan died with the dotcom bubble as it was too ambitious and therefore risky, so it was relegated to networking where it was an incredibly good job in delivering messages reliably and fast. It still am hopeful someone will do that someone will do it.