[walrus01]

> the ConnectX-6 Dx network card

I would caution people against using the Mellanox/NVIDIA network interface cards in anything they care about. The driver license is proprietary and you have to build it for your kernel as a DKMS module in a semi manual process. Makes future system updates a real bother. I realize this probably doesn't apply so much to FB internally since they build and maintain their own distro in house.

Stuff that's just a couple of years old won't even build the driver for current debian bullseye. Have to run buster? No thanks.

As with so many other things NVIDIA the root cause of this is their absurd licensing approach to open source software and drivers for their hardware.

[somebody_amzn]

(disclaimer: am an AWS employee)

Hello,

Where did you get that from? The drivers for NVIDIA networking cards are in the upstream kernel. You have the option of using newer versions from DKMS if the built-in version for your kernel is too old.

You can see the release notes for each version as present on the upstream kernel at https://docs.mellanox.com/display/kernelupstreamv512/Linux+K... for Linux 5.12 for example.

[the_only_law]

I’ve been wondering about SmartNICs recently and imagine all sorts of ideas for them. Which brand do you recommended?

> Makes future system updates a real bother.

Sounds like their GPU drivers too.

[kd913]

How about the x2 or Alveo series from Xilinx? It is basically the original definer of smartnic.

It comes from solarflare who have a long pedigree of low-latency smartnics. They used to supply Cloudflare, and also supply like 50% of fintechs/financial markets.

You can also just use openonload to accelerate your programs. In this case just doing straight linux socket programming, which can be accelerated without dpdk. Or just use the open source linux net driver if necessary.

https://github.com/Xilinx-CNS/onload

[walrus01]

I recommend Intel for 100GbE stuff. Intel's heritage of proper open source drivers (written by their own staff, in house) for FreeBSD and Linux goes back really far.

You could look at the man pages for their very expensive 10GbE server NICs in 2003-2004 and see the @intel.com email addresess of the persons who wrote them.