[fulafel]

It's interesting how much faster your laptop SSD is compared to these high end performance oriented systems. Keeping in mind that the localhost/tls-disabled number is a high bound. (Not singling out Arrow by any means, most others are slower. )

I wonder which came first, the petering off of wired network hardware perf improvement, or the software bottlenecks that become obvious if we try to use today's faster networks. 100 Mb ethernet came in 1995, gigE in 1999, 10 gigE in 2002 and gained adoption in a few years.. on that track we should have had 100gigE in 2006 and seen it in servers in 2008 / workstations in 2010. And switches / routers should have seen terabit ethernet in 2010. Today's servers(X) seem to be at about 25 GBe, and with multicore that's just 1-2 gigabits per core.

(X) according to https://www.supermicro.com/products/system/1U/

[frumiousirc]

Network is now outpacing single core performance.

The same 25 Gbps claimed by the article can be achieved with a single-threaded ZeroMQ socket. That thread will be CPU bound. To break 25 Gbps, multiple I/O threads need to be engaged.

There are already greater than 100 Gbps network links while single-core speed has stagnated for many years. Multi-threaded or multi-streamed (like in the article) solutions are needed to saturate modern network links.

[fulafel]

Yep, you can't do fast anything in software without parallelism in the modern world, as single thread performance improvements have stalled even harder than networking speed improvements. I guess my observation might be a symptom of parts of the software world staying behind at the single core performance level and only specialized applications bothering to do the major sw surgery that 10x or 100x parallellism requires.