[88e282102ae2e5b]

Sure, but I'm less worried about absolute rank and more interested in whether the language I'm using is on the order of C/C++/Fortran.

I can sell a 2x slowdown to my boss if I can show that that's the only cost of a significantly more elegant and productive language, but at 100x that's a much harder sell.

[rbehrends]

Just be careful to remember that good performance on microbenchmarks does not necessarily translate to good performance on large applications. Some factors to consider are:

1. Cache performance and locality may differ considerably for large applications. For example, a microbenchmark may perform well because it can monopolize the L1 cache in a way that is not possible for larger applications.

2. Aggressive inlining, a common factor in microbenchmark performance, may not scale well for large code bases. The tradeoffs here can be rather complex: https://webkit.org/blog/2826/unusual-speed-boost-size-matter...

3. Microbenchmarks often avoid abstraction in order to achieve high performance. This can create unacceptable software engineering costs.

[achamayou]

It's still useful as a potential filter. If someone's implemented your microbenchmark in langB and it's within 2x of langA, then maybe they're worth comparing. If on the other hand langB is 20x away from langA in microbenchmarks, you can be pretty sure that it won't ever be a good replacement.

[nostrademons]

As long as you have people of roughly equivalent skill level within their language implementing the microbenchmarks. It's not that hard to get a 10x or even 100x speed improvement in the same language when an expert rewrites the naive code that a newbie wrote.

[achamayou]

That's why microbenchmarks that have been on the internet for a while are quite nice: there's a good chance at least some experts from each language will have had a go and submitted a decent implementation.

[rbehrends]

But that's a highly (and narrowly) optimized version that took a lot of effort to build. It's not something that the average programmer you have working for you will be able to replicate. Also, some languages will not have equivalent effort put in the corresponding implementations (simple example: some languages have parallelized solutions, some don't).

[nostrademons]

I think the idea behind real-world software performance is that "if you need it, you really need to have it, but most of the time you don't need it." The benchmarks game isn't all that unrealistic for this. Most of the time, you won't care about performance at all. When you do care about performance, you'll be able to devote an experienced engineer to carefully optimize a specific hot spot, not unlike a microbenchmark. It matters how fast you can get the code to go when it's a hot spot, not how fast all your little support code runs.

My one complaint is that there's no benchmark that measures FFI performance. Realistically, if you build a system in Python or Ruby - you're going to be dropping down to C for your hot spots. And so scripting language performance on all these compute-intensive tasks is somewhat irrelevant, you really want to know how much overhead you'll incur crossing the scripting/C boundary (which, in my experience can sometimes be large enough that it wipes out all the gains of coding in C in the first place).

[rbehrends]

In my experience, if you're seeing a 20x difference, we're either already talking compilers vs. interpreters or fundamentally different implementations (e.g. one using SIMD intrinsics vs. one not using them).