[taylorbuley]

One problem I have with this approach is that it requires you to pick a threshold after which the response is "too slow." This number can change a lot over the course of an application lifetime, and would be hard to pick objectively anyways.

Median latency -- perhaps with (the smoothing-effect of) a rolling median -- would be more robust to outliers without having to resort to hardcoding of "too slow" thresholds. It would still require the human to connect the dots (e.g. median latency of >200 is "too slow") but it's an improvement on mere average response time for reasons noted.

[RyanZAG]

This actually calls for advances in analytics packages. If you could specify the threshold on a per page (or regular expression for complex requests), and have the system track and notify you of threshold exceptions, this wouldn't be too difficult to manage with some nice sliders.

Sounds like a good problem for an analytics startup to tackle.

[pla3rhat3r]

I agree with this. It's hard to gauge what is acceptable because it really depends on the application. So many other dependencies when dealing with latency and how it effects performance.

[mvolo]

hi, author here.

Unfortunately, you HAVE TO do it. If you do not set a threshold for what is acceptable, how do you determine whether or not your are providing an acceptable experience to your users?

No amount of aggregate metrics can help you answer this question unless you know whats acceptable, and what isnt - for each important set of URLs in your app.

I agree that its "hard" to do. In our own product (https://www.leansentry.com), we solve this problem by grouping urls, and using good defaults / making it easy to override the thresholds for users.

[pla3rhat3r]

There so many moving parts that this could either be a great tool to analyze data or it could open up a can of worms which could lead down the road to network re-architecture. If this takes into account best effort or SLA based ISP, Network topology, QoS, Packet Prioritization, etc then I think it could be a useful tool. Without it it's just a tool that spits out pretty pictures. If your main selling point is data then it has to be more than just what latency can show.

[mvolo]

hi there,

The post is about selecting a top level metric for monitoring website performance. One a problem is indicated, you would definitely need to drill in to figure out what part of your app is affected, when, and what caused it.

LeanSentry (our own application monitoring product, https://www.leansentry.com) does this. However, describing this was outside the scope of my post (but you can see the demo of it on the website).