Hacker News new | ask | show | jobs
by TeMPOraL 35 days ago
Good faith starts with understanding that "90" in P90 is an arbitrary number, and that the exact sampling threshold is not important to why anyone cares about that metric.
2 comments

The 99 in his TM99 is also an arbitrary number.

The more interesting part of the talk is how using what he calls "fenceposts" naturally results in bunching up against the fenceposts because even small regressions that cross the fenceposts are blocked but regressions of any size that don't cross a fencepost go unnoticed.

It's just a plain bad metric. Useful due to brevity though.

A decent metric for testing against might consist of a target mean, the sigma specifying a gaussian about that mean, a minimum sample size, and a hard upper cutoff.