[LinuXY]

Right- Though the point of the post was to dispel the fallacy that the Linux kernel can only handle 50k pp/s per core. Using RDMA effectively bypasses the kernel. He's also testing with a SolarFlare card which doesn't support DPDK, though it does support RDMA with OpenOnload. What I've found is that RDMA is the "easy" part to get right, as it's fairly simple. Not every network card is created equal with respect to how many packets it can actually pass through to the kernel however, partially in part to the kernel driver (whether it's using NAPI or not, driver efficiency, MSI-X support, interrupt coalescing) and the card itself (onboard buffer, latency characteristics, etc.) 10g cards max out at around 14Mpp/s @ 60 bytes when the kernel is involved and everything is perfectly tuned. Which should be where the card he's using falls. A generic onboard Intel card generally maxes out around 8-10Mpp/s. But both would most likely be able to hit 16Mpp/s @ 60 bytes if using RDMA in any form.

[shin_lao]

DPDK is great because it means telling to the customer "you keep your network, just buy a server with this card and we promise you unreal throughput".

If you really want maximum throughput with RDMA, I think the best is to go InfiniBand.

[Galanwe]

...Except it's dead.

InfiniBand was the way to go 5/10 years ago, when 10G Ethernet was not there.

Nowadays, most of companies that invested in IB years ago are stuck with a dead infrastructure. It costs a lot, there is very little knowledge about the techno, and most support for it is dropping (e.g. glusterfs). Sad truth is most of these old IB infra are now used for IPoIB ...

Source: been working in HFT firms implementing IB RDMA, then GBEth RDMA, now proprietary NIC RDMA

[shin_lao]

We've seen InfiniBand in HPC, it's true none of our customers in finance has it.