Hacker News new | ask | show | jobs
by StreamBright 2725 days ago
Not really, you can't have a microsecond latency talking over a network. I understand localhost microbenchmarks are nice but real life scenarios are much better.

The setup for the test:

- provision node A for being the server

- provision node B for being the client

- open X (16..16000) connections from node A and use http pipelining start to send requests to node B

I use wrk2 as the test client it is pretty amazing and looking at the latency distribution graphs.

Tools that clear winners of performance:

- https://github.com/valyala/fasthttp

- https://www.rapidoid.org

Elixir/Cowboy/Plug is in the middle range, kind of like what Techempower[1] guys saw during their tests.

[1] https://www.techempower.com/blog/2018/10/30/framework-benchm...
1 comments
pmarreck 2725 days ago
Wrk2 is not fast enough and will thus give spurious results since the bottleneck is the client.

The only measuring tool that is fast enough to accurately measure Phoenix performance is something like Tsung (which is also an Erlang app...)

link
StreamBright 2725 days ago
I am not sure what you are talking about. wrk2 can put out 7M req/s on a 16 core box. That is way beyond Phoenix's performance on the same HW type. wrk2 is widely used and accepted performance measurement tool. Again, you mentioned microseconds latency which means you are talking about localhost microbenchmarking. That is irrelevant from the production workload point of view. I have saturated network links successfully with wrk2 which is the definition of fast enough.

Interestingly there was a thread on HN previously which tools are used for HTTP perf testing:

>>> - Wrk: https://github.com/wg/wrk - Fastest tool in the universe. About 25x faster than Locust. 3x faster than Jmeter. Scriptable in Lua. Drawbacks are limited output options/reporting and a scripting API that is callback-based, so painful to use for scripting user scenario flows.

https://news.ycombinator.com/item?id=15738967

https://news.ycombinator.com/item?id=15733910

link
pmarreck 2722 days ago
alright. this looks interesting. I'll have to dive back into this space and see what's changed, I'm probably out of the loop. What's the diff between "wrk" and "wrk2"?
link