[ben509]

What do you actually do about an alarm? Okay, I just got an email, check there's no release planned, I go through the diff and, oh crap, my customers are being pwned. Now I'm going to have to drop everything and manually disable those URLs until I can contact the maintainer of this service? I guess that's the unfortunate reality when using SaaS.

I'd want a runbook feature so that anyone getting the email has a procedure to:

* check release notes to identify a false alarm

* if not identify which source files the URL is used in

* disable those URLs

* flush caches

* confirm they're disabled

* contact the service provider

* roll back when upstream is fixed

Or something like that. Man, I do not miss ops work.

[Dyaz17]

You are right, for now all the analysis should be done by the owner of the script. I'll think about adding a runbook...