Hacker News new | ask | show | jobs
by cbarrick 1164 days ago
I think this article is missing the forest for the trees.

The article is about finding the appropriate sensitivity of alerts on some signal in order to maximize the predictive value.

But you should care more about the quality of the signals you are monitoring than about the sensitivity of your thresholds.

The article mentions load-average as an example signal, but to me, that's a poor signal to monitor. Instead, if your SLO is defined for error rate, alert on error rate.

Alerts on your SLO will have a high predictive value for predicting violations of your SLO, by definition. The tunable parameter here is the time window, not the threshold. E.g. if your error budget is defined for a 30d window, you may want alerts at the SLO threshold for 24h and 1h windows.

Alert on causes, not symptoms.

1 comments

> But you should care more about the quality of the signals you are monitoring than about the sensitivity of your thresholds.

This is so true. Case in point: Growatt inverters have - like every other inverter - a maximum voltage on the grid connection at which they will shut down. They're pretty trigger happy about this and fail to take into account the resistance of the feed wire of the inverter to the (much lower impedance) grid hookup. As a result even on cabling sized properly for the interconnect they tend to falsely trigger well before the point where they should. The only way to avoid this problem is to either hack into the inverter somehow (which I've so far failed to do) or to use oversized cables (which isn't always an option).

The sensitivity is fantastic, the quality of the signal is hopeless. Obviously they err on the side of caution but the margin is so ridiculously large that you end up losing a lot of usable power for no reason at all. At least it should allow for either a resistance for the interconnect to be specified so that it can take into account the voltage drop across that wire, which at 10A is appreciable for even short runs of fairly beefy cable.