Hacker News new | ask | show | jobs
by jarofgreen 774 days ago
I had a similar concern about a custom monitoring script that was regularly writing out a metrics file in Prometheus format to be picked up by the Prometheus node monitor. What if the script breaks?

So I set the script to also write out a metric that was just the time stamp the metrics were last updated. Then it was simple to set up an alert in Prometheus - I can't access the config now so you'll have to look it up yourself, but it was basically "alert if metric less than now minus a time gap"

1 comments

I've been using it for some time and it works well.