I was either unaware (or had forgotten that) he had taken a second stab at the benchmark. I do hope he updates the document with the root cause of the enormous variance in response time.
I note with interest that the two frameworks which are faster than Phoenix (in both throughput and latency) are Gin, written in Go, and Play, written in Scala. In this run of the benchmark, they also have more consistent latency than Phoenix, although the author notes a suspected problem with his Erlang installation; previous runs had consistency better than Play, but still not as good as Gin.
I note with amusement that:
> Play consistently appeared to generate hundreds of socket read errors (see the detailed output), so I believe it should probably be semi-disqualified from the results for now.
I'd be interested to see another JVM framework tested. Play is really rather complicated, and it's written in Scala, which is also rather complicated. I'd love to see a comparison with a Sinatra-for-Java type framework like Spark, or perhaps Undertow (since it does so well on the TechEmpower benchmark). Oh well, maybe it's pull request time ...
Until he figures out what was up with his terrible Elixir consistency numbers, I would treat the second round of results with some suspicion. Maybe something changed in Phoenix/Elixir/Erlang, or maybe something's up with his machine.
If you do submit a PR, I do hope that you have the time and gumption to re-run all of his round 2 benchmarks, in addition to any additional ones you care to see. :)
I was either unaware (or had forgotten that) he had taken a second stab at the benchmark. I do hope he updates the document with the root cause of the enormous variance in response time.
Edit: Good find, BTW. Thanks for digging that up.