[grive]

This patch brings XDP RX to 4.2Mpps which is nice.

In 2015 high-speed networking was targeting 10G line rate, i.e. 14.8Mpps. Today the NICs are 100G (42Mpps), next year it will be 400G.

I'm not sure how the kernel is expected to follow with the pace. Those NICs will offer offloads, so static network functions should be accelerated without involving the CPU. Still, it seems software-based networking will never be able to catch up.

DPDK and userland solutions in general are able to scale better. At least for DPDK, depending on the workload an L2 forwarding function can do 14.8 Mpps on a single core, and supposed to scale linearly (so 4 cores for a 100G L2 fwd?).

However I've yet to see a TCP implementation able to scale linearly. DPDK in used generally for middleboxes, not endpoints, where scaling TCP has different challenges (handling a socket state with a single RSS dispatch vs. two RSS dispatch I guess).

So what does it say about the future of SDN? Will it be able to keep pace with hardware network functions? A pure kernel-based approach would be nice but it seems impossible. On userland side the industry is not there yet in open-source, as far as I know. There are probably proprietary stacks able to do this, but I don't know them.

[diegocg]

There is nothing that user space solutions do that can not be implemented in kernel space. From what I understand, the Linux kernel networking stack is not optimised to process packets in bulk, which is the only way to deal with high speeds nics https://lwn.net/Articles/629155/

[uluyol]

True, but we have a kernel networking stack with a certain interface and a rich feature set. The kernel cannot break that API.

In contrast, a new userspace stack doesn't need to worry about backwards compatibility or supporting every use case.

So yes, the kernel can do just as good, but it's harder.

[miltondts]

This is probably the only way to have fast software in today's hardware.

Unfortunately the popular languages are all about single element at a time, and pointer chasing with the compiler having a hard time vectorizing operations and reorganizing data structures. I think most compilers don't/can't even do this because there are not enough guarantees. The only systems that actually do this are DBMSs.

[MaxBarraclough]

> compiler having a hard time vectorizing operations and reorganizing data structures

Reminds me of a recent blog post on exactly this topic. Apparently the Jai language might make some progress on this point.

• https://blog.royalsloth.eu/posts/the-compiler-will-optimize-...

• Discussion: https://news.ycombinator.com/item?id=27010965

[drewg123]

I hate to nit-pick, but 100GbE line rate is 144.8Mpps, and 400G line rate is 579Mpps

[grive]

You're right to nit-pick! It is rather embarassingly wrong (10G -> 100G should be a pretty obvious 14.88Mpps -> 148.8Mpps...). Unfortunately it seems I cannot edit the comment so I'll leave the error for posterity ^^. Still, thanks!