Hacker News new | ask | show | jobs
by matttproud 1754 days ago
I ran multiple search queries. I wouldn't be so dumb to post a comment like this here without having done my homework. The best I found after trying numerous keyword permutations was https://salsa.debian.org/benchmarksgame-team/benchmarksgame, but this did not appear to contain all of the benchmarks' source, just the source embedded in HTML, which is specious at best. This repository looks mostly like frontend HTML and chrome, not a SUT, executor, nor even the sub-test code.

At the very least, I couldn't realistically re-run some of the example benchmarks from the source embedded in the HTML, because they did not include vendoring/version information for external packages they depend on. That made me doubt the provenance of https://salsa.debian.org/benchmarksgame-team/benchmarksgame.
4 comments
igouy 1753 days ago
> This repository looks mostly like frontend HTML…

It is mostly frontend HTML — the benchmarks game project deliverable is a (now static) website.

> I couldn't realistically re-run some of the example benchmarks from the source embedded in the HTML, because…

People have taken the simplest thing that will work approach: select/copy the program source code from the website, paste into a text editor and save; then build & run adapting the commands shown on every program page.

link
igouy 1752 days ago
Perhaps these other projects are a better match with what you expect to see —

https://programming-language-benchmarks.vercel.app/

https://github.com/kostya/benchmarks

link
igouy 1753 days ago
> … just the source embedded in HTML…

"Where can I get the program source code? — zip'd program source code"

line 11 ? in the README

> … vendoring/version information for external packages they depend on…

If the programs don't build/run with the latest GA external packages, they will be shown as "Make Error" "Bad Output" "Failed" until someone updates them.

link
matttproud 1753 days ago
It would do the project’s web site a big service to actually link prominently to this repository, if this site is in fact canonical.

A zipped source code archive inside a Git repository is very surprising (violation of principle of least astonishment). No wonder I couldn’t find the benchmark programs’ source; they aren’t even indexed for code search due to residing in a zip file.

link
igouy 1753 days ago
And, of course, there was a time when the website did "link prominently to this repository" — back before July 2020, when "new source-code was usually measured and shown on the website within a few days."
link
igouy 1748 days ago
> … made me doubt the provenance…

To address that concern, there's now a link in the homepage banner.

link