[jreichhold]

Very glad that Jeff wrote this up and he deservers mad credit for documenting something that has been frustrating me for years. People don't realize how hard doing this right is and discount the hard work that goes into making large systems scale in a stable manner.

It isn't just load testing but more that the whole system should be considered suspect. If you don't act defensively at all steps you will be hosed by something you thought will never happen. Just had a good talk about this last weekend. Memory, TCP, and all other rock-solid things can and will have issues in large systems.