[emeryberger]

You should never do this. Best-case favors outliers and does not represent expected performance, which is what we care about. Just because the stars happen to align one time doesn't mean you report that run.

Consider the following runs of two systems:

system A: 10s, 10s, 10s, 10s, 10s, 10s, 10s, 5s

system B: 6s, 6s, 6s, 6s, 6s, 6s, 6s, 6s

Which one is faster? (Hint: don't say system A)

[alephnil]

You will in practice hardly ever see outliers like you descrivbed in system A, where one run is significantly faster. You will often see cases where one run is significantly slower. The reason could be things like cache misses, swapped out code, some bad code path happening etc (all these on very different timescales). These things tend to happen only occasionally, so the reversed case from your example A (seven five second runs and one ten seconds run) is more pluasible. Because such factors tend to be things you can't easily control, taking the minimum is a good approximation when optimizing a code snippet as opposed to the whole program.

[to3m]

But this ignores the other way of looking at it: if system A is slower, how come it managed to run more quickly?