I am planning a big ruby benchmarking blog post in the next week or so. I will definitely add Jubilee to the existing list of rack servers to test with!!!
I plan on benchmarking on thin, unicorn, puma, WEBrick, torquebox 3, torqbox (torquebox 4), trinidad, passenger, and I guess now Jubilee.
I was also thinking of doing some EventMachine based stuff like goliath and whatever else I can find in that space.
From my early benchmarks last weekend torqbox is the fastest and pure rack is a lot faster than sinatra or rails. Sinatra is not nearly as fast as I thought it was compared to pure rack. A simple thin server running ruby 1.9 got about 1500 req/sec for hello world using Sinatra. Torqbox and jruby brought that to about 5000 req/sec. Plain rack was about 6500 req/sec on ruby 1.9 and thin and on torqbox it was 10,250 req/sec. For reference, my golang hello world was getting 10,500 req/sec.
I'm very curious where Jubilee will land in that mix.
One more ruby framework that might be interesting to test would be grape. https://github.com/intridea/grape Thanks for the work you've done with bench-micro. I was not aware of all of those frameworks.
Yep, I did. I'm sure they are some fiddly bits that could have driven it a little higher, but if you look at the tech empower latest bench, the torqbox json rack test is within spitting distance of golang, so I don't think my results are that far off.
Anyone willing to craft a Jubilee test implementation for us [1]? It would be nice to see the Torqbox versus Jubilee grudge match (which should echo the Undertow vs Netty title fight).
Since both Jubilee and TorqBox are easy to install, I did a quick test of both on my machine with the TechEmpower JSON app to get a feel for the relative performance - https://gist.github.com/bbrowning/8244516
My laptop gets 19k req/s with Jubilee and 118k req/s with TorqBox on the same Rack app. While Java on Vertx does fare very well in benchmarks, it's incorrect to assume this directly translates to other languages and frameworks written on top of Vertx also having the same high performance.
I guess you didn't warm up the JVM before running the tests, since the Jubilee/Vert.x stack is more heavy than the undertow counterpart, it takes longer for the JVM to apply the optimisations. I also did some benchmarks for Torqbox and Jubilee, you can find the results here: https://github.com/isaiah/jubilee/wiki/Benchmarks-against-ot...
Torqbox won the rack test, and Jubilee out performed in the rails one.
I did warm up the JVMs, actually. Those Rails numbers are far too low for TorqBox. Perhaps the best thing to do is add this to the techempower benchmark suite so a 3rd party can run and present the results.
Edit: The demo chat application looks totally awesome! http://192.241.201.68:8080/