[pdimitar]

You should read a bit more on how the various popular benchmarks manage their different code bases and not just quickly believe the graphs. There's a lot of story behind them.

Several languages, Elixir included, are misrepresented due to badly written and non-idiomatic code (caveat: this was true a while ago, I haven't checked lately though). And when its users try and open a PR they are shot down with "We don't allow cheats". No cheats though, just very normal language best practices.

So I disagree with you here. I maintained a ton of Rails and several Sinatra apps and to say they are on the level of Elixir makes me wonder what kinds of hosting budgets we both had access to and were they really so hugely different.

(Sinatra was actually very decent but it being Ruby it doesn't scale well and Erlang/Elixir work much more stably until they start having problems compared to it.)

Again, don't just trust the popular benchmarking suites. You will quickly discover they usually either have an agenda (like the Crystal language creators always showing how quicker it is compared to Elixir, and refuses any Elixir PRs -- seriously no idea what the Elixir community ever did to them) or only pay lip service to representing all languages fairly (like TechEmpower does).

(Edited because I wrote the first version after a nap and it came off sharper than I intended.)

[bhauer]

There is an implication here that we (TechEmpower) have rejected PRs that would make the code in our Elixir tests more idiomatic. And if I understand you correctly, when we did so, we explained that the implementation did not comply with the spirit of our tests. To be clear, we do reject pull requests that we believe are cheating (e.g., caching results or other means to avoid specific workloads called out in the test requirements).

Can you point to which Elixir-related PR [1] we rejected on that basis?

Some of our most recent Elixir commits were contributed by a gentleman named Michał Muskała. For example, here are some he characterized as "minor optimizations" [2]. Since he is a one of 16 people on the Elixir language organization page [3], I take his contributions as representative of idiomatic code or at least what the organization wants to represent as idiomatic. Do you have a reason to disagree?

[1] https://github.com/TechEmpower/FrameworkBenchmarks/search?q=...

[2] https://github.com/TechEmpower/FrameworkBenchmarks/pull/3848

[3] https://github.com/orgs/elixir-lang/people

[pdimitar]

Thanks for chiming in.

Michał Muskała is a core contributor and his opinion weighs a lot so I am really glad you guys are working with him.

I did check out the PR and his changes seem to be on par with what I've seen in other repositories used in TechEmpower's benchmark. Not cheating, just making sure not to use extra baggage that can severely hamper a framework's performance.

As for rejected PRs, I looked but couldn't find what I roughly had at the back of my head. It was a while ago and since I can't find it now I will have to concede that I remember incorrectly. (Ouch.)

As I pointed out, my info was outdated. I see things have improved in the meantime, which is good news!

Thank you for being responsive and transparent. Much appreciated.