Hacker News new | ask | show | jobs
by igouy 1068 days ago
You seem to be looking at the wrong source code.

The "Energy Efficiency across Programming Languages, SLE’17"authors provided this repo —

https://github.com/greensoftwarelab/Energy-Languages/blob/ma...

https://github.com/greensoftwarelab/Energy-Languages/tree/ma...

For a single outlier (regex-redux) there's a 12x difference between the measured times of the selected C and C++ programs.

Even so, that mostly messes up the results because the arithmetic mean is used rather than the median.

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

1 comments

Did the article use the wrong link? I used the link in the article. The C++ code from your link looks the same as what I saw.

Your bottom link shows the same data I see using the link in the article: the C++ is ~50% slower on spectral-norm.

I expect that the 12x outlier is because someone did something very non-optimal in C++.

Yes, the link grabbing seo claims the benchmarks game has something to say about Java and energy-efficient — in fact the benchmarks game does not measure energy use.

No, the link grabbing seo provides the correct source for Table 4 — "Source: Energy Efficiency across Programming Languages, SLE’17" — and google finds the article:

https://greenlab.di.uminho.pt/wp-content/uploads/2017/09/pap...

> looks the same

Please look again.

> the 12x outlier

    #include <pcre.h>
    vs
    #include <boost/regex.hpp>
Yeah so the 12x gap in regex-redux is measuring the speed of two completely different libraries, and obviously boost is what’s slow. It’s not a language difference, nor even primarily an issue with means vs medians. The issue with regex-redux is slightly more of comparing apples to oranges.

I assume you’re referring to regex-redux, but I was confused for a minute there because you linked directly to the C++ code for spectral-norm, not regex-redux, and spectral-norm doesn’t include a regex lib. I had to go looking around for the includes you’re talking about, and the Benchmarks-Game version has multiple C++ entries. There are C++ entries that use pcre, re2, and boost/regex. The C entries all use pcre except the winner, who uses pcre2.

> For a single outlier (regex-redux) there's a 12x difference between the measured times of the selected C and C++ programs.

BTW, I dont know exactly what they’re referring to there. The time ratio of the Benchmarks Game fastest C version to the slowest C++ version is over 16x. The ratios in the SLE paper of energy and run time are a bit lower than 12 but they also have only one sample, and it uses boost, so I don’t see why regex-redux with boost is useful in any perf or energy comparison other than to serve as a mild warning about using boost.

There happens to be a ratio of almost exactly 12x between the slowest C++ version (g++#3 boost) and the fastest C++ version (g++#6 - pcre2). That’s C++ vs C++. Maybe that’s what they were talking about, or maybe the perf numbers have changed out from under the commentary.

The ratio of times of the fastest C to the fastest C++ time with regex-redux is 1.4x and they both use pcre2. That seems like a more reasonable place to start.

> … not … primarily an issue with means vs medians

We're comparing averages, why would we bother so much about the cause of an outlier.

> you linked directly to the C++ code for spectral-norm

You had linked to the wrong C and C++ code for spectral-norm, I linked to the code that was actually used.

> The time ratio of the Benchmarks Game fastest C version to the slowest C++ version is over 16x.

Again, you seem to be looking at the wrong repo.

The authors of "Energy Efficiency across Programming Languages, SLE’17" provided this repo —

https://sites.google.com/view/energy-efficiency-languages

https://github.com/greensoftwarelab/Energy-Languages

I was talking about the Benchmarks game repo code. (But it doesn’t even matter because the SLE is equally bad.) And I was not comparing any averages, I was pointing out that use of boost is the entire reason the average is skewed and misleading. You seem to be misunderstanding my comments.
> I was talking about the benchmarks game repo code

Why? The benchmarks game doesn't show energy use of programs.

> use of boost is the entire reason

As you know, the fastest C++ program shown on the benchmarks game website uses pcre2.h.