[angusgr]

You're entirely correct on both counts. In fact, all 3 of those reasons given on that page are good ones.

However, this was still posted under the title "Which programming language is the fastest?" - so I think it's worth exploring in exactly what ways that is or isn't useful information. Just as their own reason #3 says - "To show how difficult it can be to make meaningful comparisons"

I'd also question exactly how useful reading toy benchmark solution code is to real problems. The code to the benchmark problems is often nicely written, but if I have a problem to solve then wouldn't I be better off looking for samples of readable code that solve problems related to my specific problem domain?

[igouy]

> worth exploring in exactly what ways that is or isn't useful information

Now consider how you could use the varying performance on different tasks, and the varying performance on the same task of programs written in the same programming language, as examples (as a resource) for exploring (and showing to others) in what ways you think that is or isn't useful information.

If we just consider those "Which programming language is the fastest?" pages, then I think - simply getting someone to look away from a single number ranking (the median) and notice that for some language implementations the performance range is relatively small but for others the range is enormous - is useful.

> I'd also question exactly how useful reading toy benchmark solution code is to real problems.

Someone's told me that n-body pretty much is what they get paid to work on. Someone's told me that k-nucleotide and reverse-complement are pretty much is what they get paid to work on. The question as usual is who's "real" problems?

> wouldn't I be better off looking for samples of readable code that solve problems related to my specific problem domain?

Perhaps - and yet for some people what they see in those tiny tiny programs will be enough for a personal opinion about whether X is "the most human-readable".

[angusgr]

These are all good points. I must confess, when I first looked at some shootout problem code a few years ago I remember being in violent disagreement about how some of the solutions were implemented (ie thinking "If I were solving this problem in a non-toy way I wouldn't do it that way, that's a total hack for benchmark purposes"), but it looks like they've evolved a fair bit and I can't find the same complaints I had then - so kudos for that. I need to recalibrate some of my opinions on that front. :)

Given that this sounds like its your own project, I have a vague idea that you might find interesting: Is there a way to present the results in an additional way which is framed initially as "look at the different ways to solve subproblem X" rather than "look at the graph of performance differences"? It could be an interesting approach to have an alternative additional interface that makes browsing the sample code (maybe with annotations?) front and centre, with a focus on programming techniques as a major differentiator, with scores as an adjunct. At present the visitor is encouraged to start from benchmark scores (either for everything, for a problem, or for a language) and from there move to source code later, but there might be a way to frame it as "let me flip through solutions to problems like X and tag the ones I like the look of, then let's see how they performed"

It's a pretty vague idea, and I don't have much of a handle on how it'd work from a UI perspective, but its seems like it might head off some of the kneejerk responses you get from kneejerkers like me ;), plus it could help encourage some of the uses you're describing in your post.

[igouy]

> browsing the sample code (maybe with annotations?)

At present, even from the home page, source code is only 2 or 3 clicks.

Who would write those annotations? Currently there are 1100 programs, but they change as new programs are contributed and others removed.

Who would do the work to create a series of blog posts discussing the development of each and every program?

http://adventuresinmercury.blogspot.com/2011/04/shootout-spe...

[angusgr]

At present, even from the home page, source code is only 2 or 3 clicks.

Yes, but it's always at the "bottom" of the interface when you drill down. For instance, all of the content is there to make an interface which allows side-by-side comparisons of the implementation of the same problem in different languages. At the moment, you can do that but it involves opening two tabs and clicking forward and back through each language.

Who would write those annotations?

The annotations were just a throwaway idea. I noticed some of the programs are very well commented, some aren't. Probably good commenting & structure covers a lot of it.

Who would do the work to create a series of blog posts discussing the development of each and every program?

I'm confused by the rhetorical questions. You seem to be suggesting noone would ever do the work to explain the creation of a program, then you link to a blog post which does just that.

[igouy]

> opening two tabs and clicking forward and back

Or opening two browser windows and looking side by side - by design the page layouts are narrow to enable side by side comparisons (often I've reformated programs to 80 char width), it wasn't that I foresaw the emergence of small screen iPhone browsing ;-)

> suggesting noone would ever do the work to explain

When something seems peculiar I've pursued contributors for some text to explain what's going on with their program - so my thoughts about this are based on my experiences

http://shootout.alioth.debian.org/u32/program.php?test=threa...

> a blog post which does just that

As an example of something that's been done well, and done independently of the benchmarks game.

Anyone can make that kind of analysis and publish on their blog.

The benchmarks game is a starting point and a source of questions, rather than a definitive answer to every question.

[igouy]

> all of the content is there to make an interface which > allows side-by-side comparisons

Updated the Help page

http://shootout.alioth.debian.org/help.php#twowindows