[michaelt]

The solution is "don't apply untested upgrades to critical servers at 3am" :)

If you must do such upgrades, solutions include hot standby hardware, IPMI, an on-site tech with a screen and keyboard, or moving everything to the cloud.

[Scramblejams]

Of course, but all those things (including upgrading during business hours) require budget. So, back to my original question...