[kcarnold]

I see they monitor a complete process lifetime, not just the active workload. A quick grep of the paper didn't turn up anything discussing this. And it would seem to hurt dynamic languages and runtime-JIT languages a lot. Perhaps the active workload takes long enough that the transient is washed out. Anyone have insights?

[fnord123]

If you want to tease the startup out, then you can run the game with a null or extremely small input set and then discount the time it takes to run a small/null workload from the actual run.

But really, if they hurt, then they get hurt. There's only so many knobs you can turn and so many runs you can run before you have to assemble a table and submit your paper. They put all the code online so you can try it yourself.

[igouy]

> Perhaps the active workload takes long enough that the transient is washed out.

That does seem to be the case with some-of the benchmarks game programs --

http://benchmarksgame.alioth.debian.org/sometimes-people-jus...