[trashhalo]

I keep wanting to be hyped about elixer but the performance benchmarks confuse me. If you look at the tech emperor benchmarks[1] phoenix makes the list at #46 registering 16% of the performance of the top framework. I'm willing to sacrifice performance for readable maintainable code but it just surprises me that its that slow.

Anyone know whats going on there?

1. https://www.techempower.com/benchmarks/#section=data-r17&hw=...

[hnra]

Simply looking at its position in that list will tell you nothing without looking at what is above and below. I don't use Phoenix or Elixir but from what I understand Phoenix tries to be for Elixir what Rails is for Ruby. That is, it is a full web framework. The number one position on that list is a "Asynchronous PostgreSQL driver".

This would be a more apt comparison:

https://www.techempower.com/benchmarks/#section=data-r17&hw=...

[rehemiau]

There are more reasons, but

- better configuration (tweaked to this benchmark / hardware). Bigger communities will try harder to tweak their benchmark config. The Phoenix one is probably just the default one or slightly tweaked. Is the SQL connection pool size ideal? Are all default Phoenix-added Plugs ("middlewares") needed here? Is the BEAM virtual machine config / flags ideal? Would using OTP releases be beneficial?

- Elixir ecosystem is constantly improving the performance. The VM, language and packages versions should be updated. Elixir 1.6.5, Phoenix 1.3.2, and Ecto 2.1 are not recent.

- Abstractions used (and amount of them, so amount of work done at run-time). Compare the Fortunes "handler" implementation in vertx-postgres [1] to the Phoenix one [2]. Raw SQL vs generated query, almost bare DB connection pool vs Repository abstraction, and it's just the "handler". But you wouldn't want to maintain a lot of low level code.

- Typical functional language overhead. Copying (copying the conn when modifying it, Enum.sort on a list etc) is more expensive than modifying in-place. Again, functional code will be easier to maintain.

But overall I think in the particular benchmark you linked, Phoenix isn't that bad. Latency is pretty low and consistent (0.5 ms min and 9.9 ms max (!)) thanks to the BEAM pre-emptive scheduler. And BEAM will scale well vertically.

1. https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast... 2. https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...

[mattferderer]

I don't know the exact specifics here but I know a lot of communities that rank high invest time contributing into the open source Tech Empower tests for their framework. This way they can tweak & make sure their stack runs at optimal speeds.

At the same time, Elixir (and Erlang) are not meant for raw speed. It is best used for real time communication, lots of users & handling errors. At least that is what I have read.

[djm_]

>Anyone know whats going on there?

Yup, many things.

It's worth reading this very long thread about exactly this, from 2016. Look for Sasa Juric and Chris McCord's comments in particular. [1]

tl;dr benchmarks are hard, not all benchmarks are implemented well, Elixir folks haven't heard back from Tech Empower re: details of errors rates etc. It's an unfair analysis, and not just for Elixir.

[1] https://elixirforum.com/t/techempower-benchmarks/171/44