Hacker News new | ask | show | jobs
by acconsta 3907 days ago
The author also points out that some of the benchmarks poorly represent real workloads:

"Bottom up (since the worst offenders are now first),

binary-trees is silly since it measures allocation speed for a case that simply doesn't exist in real code;

thread-ring is basically insane, since nobody ever bottlenecks like that;

chameneos-redux's C++ implementation is ridiculous. The C is not so ridiculous, but you still have the problem that basically every language in the top few spots does something completely different;

pidigits tests whether you have bindings to GMP;

regex-dna tests a regex engine on a small subset of cases (arguably the first half-acceptable benchmark);

k-nucleotide tests who has the best hash table for this particular silly scheme, and they don't all even do the same thing (eg. Scala precompacts, like my new Rust version);

mandelbrot is kind'a OK;

reverse-complement would be kind'a OK if not for a few hacky implementations (like the Rust);

spectral-norm is kind'a OK;

Haskell basically cheats fasta (which is why I copied it);

meteor-contest is too short to mean anything at all;

fannkuch-redux is probably kind'a OK,

n-body is kind'a OK.

So maybe 5/13 are acceptable, and I'd still only use 4 of those. I think if looking at mandelbrot, spectral-norm, fannkuch-redux and n-body you can argue the benches are a reasonable measure of peak performance. However, these cases are also all too small and simple to really be convincing either, nor is it particularly fair (where's NumPy for Python?)."

https://users.rust-lang.org/t/blog-rust-faster/3117/12?u=acc...

1 comments

>>bad at simulating real workloads<<

Have you looked at the benchmarks game website?

Please show where the benchmarks game website claims that those tasks simulate "real workloads" (whatever that means).

You will see "Your application is the ultimate benchmark" and "These are just 10 tiny examples" and …

http://benchmarksgame.alioth.debian.org/dont-jump-to-conclus...

I apologize if I gave the impression I was claiming otherwise. It's just context for the discussion.
Veedrac "points out" his likes and dislikes, that doesn't mean his likes and dislikes are "The Truth".

When Veedrac dismissively tells you - "pidigits tests whether you have bindings to GMP" - you should ask why he hasn't told you that the measurements can be different even when all the programs use GMP; you should ask why he hasn't told you that the measurements also show the difference for the same language implementation when programs do and don't use GMP.

etc etc