[kasey_junk]

If what you are suggesting is that latency measurement is difficult but what is probably most interesting in the context of http service libraries, I completely agree.

My major issue was, if they had run this exact same test and reported in "request overhead in seconds" would be largely not valuable at all because it doesn't tell you nearly enough information to determine if there has bean a meaningful latency regression.

With throughput, its likely not as valuable in real usage, but the single stat does tell you there was a throughput regression.

So I think we agree that this isn't a meaningful regression, I just disagree that changing how you report the number would be valuable.