[munificent]

> Short-lived interpreted / level-1 JITted code is not interesting at all from benchmarking perspective, because it will be compiled fast enough to doesn't matter in grand scheme of things.

This is true for servers but extremely not true for client-side GUI applications and web apps. Often, the entire process of [ user starts app > user performs a few tasks > user exits app ] can be done in a second. Often, the JIT never has a chance to warm up.

[blacklion]

If it is done in literal second, why will you benchmark it?

In such case you need "binary" benchmark: does user need to wait or not? You don't need some fancy graphics, percentiles, etc.

And in such case your worse enemy is not JIT but variance of user's hardware, from old Atom netbook to high-end working station with tens of 5Ghz cores. Same for RAM and screen size.

[munificent]

> If it is done in literal second, why will you benchmark it?

The difference between one second and two seconds can be the difference between a happy user and an unhappy user.

> You don't need some fancy graphics, percentiles, etc.

You don't need those to tell you if your app is slow, you need them to tell you why your app is slow. The point of a profiler isn't to identity the existence of a performance problem. You should know you have a performance problem before you ever bother to start your profiler. The point is to give you enough information so that you can solve your performance problem.

> your worse enemy is not JIT but variance of user's hardware, from old Atom netbook to high-end working station with tens of 5Ghz cores. Same for RAM and screen size.

Yes, this are a real problem that client-side developers have to deal with. It's hard.