Depends on standard - within the clued-in performance community, yes, but there are major, major companies still pushing averages and that causes a lot of people, particularly those without much stats / engineering background, to expect it everywhere.
To use one example which is prevalent throughout marketing, advertising, etc. Google Analytics reports only averages – this makes the results unreliable enough that I'm now advising people to simply pretend that field does not exist as it's completely untrustworthy. Awhile back I blogged about an example where 3 samples out of 200K threw the average off by a full order of magnitude: http://chris.improbable.org/2012/05/18/google-analytics-dece...
Very interesting, thank you. I especially like the replies from the Google analytics team, 8 months apart, that both acknowledge the issue and say they'll fix it...
Also that in addition to not having fixed it, accurate stats are apparently less of a priority than, say, a gigantic fixed-position toolbar. That's been disappointing…
Author here. The 99 or 95 percentile is a much better metric! We also make the case for the industry standard Apdex or our derived metric, sat score. These are becoming more and more in use by APM tools like us or New Relic.
Unfortunately, many existing tools and people who use them still look at latency aggregates and often make incorrect assumptions.
To use one example which is prevalent throughout marketing, advertising, etc. Google Analytics reports only averages – this makes the results unreliable enough that I'm now advising people to simply pretend that field does not exist as it's completely untrustworthy. Awhile back I blogged about an example where 3 samples out of 200K threw the average off by a full order of magnitude: http://chris.improbable.org/2012/05/18/google-analytics-dece...