Hacker News new | ask | show | jobs
by growse 844 days ago
The people feeling the pain of (and paying for) the expensive data swamp are often not the same people who are yolo'ing the sample rate to 100% in their apps, because why wouldn't you want to store every event?

Put another way, you're in charge of a large telemetry event sink. How do you incentivise the correct sampling behaviour by your users?

2 comments

Don't let the user pick the sampling rate. In Honeycomb land this is called the EMA Dynamic Sampler.

https://docs.honeycomb.io/manage-data-volume/refinery/sampli...

You should never need to sample telemetry data.