[pron]

My point is that the website offers a very low coverage of the codepaths that affect language performance, and all it says is that the coverage is incomplete without saying just how incomplete it is. People who don't know that, for example, optimisation in some languages can vary greatly depending on whether there's one compilation unit or several (especially since most/all examples have one compilation unit while virtually all programs have many) and that memory management depends heavily on the variety of object sizes and may degrade over time (especially since the examples all run for a short time and have a low variety of object lifetime and size while most real programs are different), can come to very wrong conclusions.

Sorry for yet another analogy, but it's just like me telling my boss that my test suite passes but it's incomplete without telling him that I've tested only 20% of the program's functionality.

The website should at least explain just how skewed the results are in favour of conditions that arise in small, short-lived programs without (competitive) concurrency and that larger and/or longer-lived and/or concurrent programs can exhibit very different behaviour. These conditions aren't a small matter. They're among the primary motivations for the huge investment over the last few decades in moving GCs and JIT compilers.

[igouy]

> People who don't know … can come to very wrong conclusions.

Always.

> … website should at least explain…

Whatever the explanation, it would never be sufficient for you.

As you said -- "If you're asking what multi-lingual benchmark suites offer good coverage - I don't know."

[pron]

A website that doesn't give people the relevant context for the data it offers is not a good website, especially when the data cannot be understood without that context and the context is not at all well known. That people may misinterpret the information if it were given to them doesn't change that.

> Whatever the explanation, it would never be sufficient for you.

Since I work on compilers and runtimes I know just how small the coverage offered on that website is. For me, no benchmark suite may ever be enough. Still, offering some of the information that is needed to put the results into context would be better than offering none.

Again, I'm not saying that the website is intentionally misleading and I have nothing against the people behind it. They may have a very good reason for why it's of such low quality. I'm just pointing out that it is.