[JoshuaRLi]

It isn't surprising to me that instruction counts as a performance metric would have less variance than wall time. Did you know that the environment size can actually affect wall time?

> We see that something external and orthogonal to the program, i.e., changing the size (in bytes) of an unused environment variable, can dramatically (frequently by about 33% and once by almost 300%) change the performance of our program. This phenomenon occurs because the UNIX environment is loaded into memory before the call stack. Thus, changing the UNIX environment size changes the location of the call stack which in turn affects the alignment of local variables in various hardware structures.

From https://www.inf.usi.ch/faculty/hauswirth/publications/asplos....

[vlovich123]

That's exactly my point! The only thing that matters is wall clock time so if cache layout is making a 300% difference to wall clock, focusing on instructions counts will mislead you on the most impactful optimization to make (i.e. cache layout optimizations won't show up in instruction counts).

And yes. I'm aware of that result because of Professor Berger's talks on Coz & the other work he's done in this space.

[JoshuaRLi]

> The only thing that matters is wall clock time so if cache layout is making a 300% difference to wall clock, focusing on instructions counts will mislead you on the most impactful optimization to make

Ahh, very well said!