|
|
|
|
|
|
by calmtech
1305 days ago
|
|
|
We noticed that one line of byte code was crazy hot. We traced it back to the code, and realized it was a special case fallback. They were summing 10mb of 8 bit encoded integers, in the 99.5th percentile of traffic, so no one noticed. It was using 10% of fleet compute. We changed the encoding to 64bit; the data size stayed constant,
the 99.5%tile latency plummeted, and I got a bonus. Took a week. My contribution to climate change. A lot of stuff can hide past 99%. |
|
|