[optimali]

Would be interesting to try and fit a distribution to error rates (some type of counting process) and then monitor the probability of having the occurred number of errors (with in some period of time). Then low probability events might indicate an outlier.

[diab0lic]

We have another component of the same outlier detection system that does this type of fitting, identifying low probability events using a Bayesian model + Markov Chain Monte Carlo. It hasn't gained nearly as much traction internally (yet) as the clustering approach here.

[optimali]

Cool. Have you found the MCMC approach to be less robust than clustering? It seems like this may be the case due to probability model assumptions.