[hyperpape]

The basic problem is that if the compiler handles your code in an unusual way in the benchmark, you haven't really measured the two implementations against each other, you've measured something different.

Dead code elimination is the most obvious way this happens, but you can also have issues where you give the branch predictor "help", or you can use a different number of implementations of a method so you get different inlining behavior (this can make a benchmark better or worse than reality), and many others.

As for runtime, if you're creating a library, you probably care at least a little bit about alternate runtimes, though you may well just target node/V8 (on the JVM, I've done limited benchmarking on runtimes other than HotSpot, though if any of my projects get more traction, I'd anticipate needing to do more).