[chrisseaton]

> PLs arguments are very rarely falsifiable

You mean PL design, don't you? Performance work is very empirical. If my new type of inline cache is better then I need to prove that and it's falsifiable (using benchmarks, which I admit aren't ideal).

[chrisseaton]

In fact! I can falsify your claim that PL research isn't falsifiable by using some PL research!

http://cis.upenn.edu/~cis501/papers/producing-wrong-data.pdf

> Of the 133 papers published in the surveyed conference pro- ceedings, 88 had at least one section dedicated to experimental methodology and evaluation

[fjsolwmv]

That puts it on par with psychology or sociology which has a poor reputation in the falsifiability department

[chrisseaton]

I think most of the other papers have non-experimental proofs, so it's not that they do nothing rigorous.

And anyway the claim was 'very rarely', when it's actually a majority of cases.

[fjsolwmv]

Still, the benchmarks are usually toys and it's pretty easy to find contradictory benchmarking reports. Very few projects do online benchmarking comparisons of different design choices on real workloads