[guelo]

Weird that they didn't say what caused the power outage and what the mitigations are for that.

[sh4na]

If it's a data center owned by a third party, they probably can't talk about it.

[jlgaddis]

  RFO: A squirrel climbed into a transformer
       and a short time later they both blew up.

[gsibble]

I'm also confused about how the racks would lose power. Surely they had UPSes.

[abrookewood]

Generally speaking, I'd recommend AGAINST running UPSes in racks that are managed by top-tier data centres. I've had way more trouble with UPSes misbehaving than I ever have with data centres losing power. EDIT: I'd also point out that 2 hours is a long time to be running on in-rack UPSes. I've usually seen them designed to withstand about an hour, but not much more.

[WatchDog]

The power outage was only brief, enough to halt the servers but, much less than the 2 hour outage window.

[ams6110]

UPSs don't always cover everything. There are systems that are considered critical that are on UPS, and others that are considered restartable that might not be. There are a lot of tradeoffs in a data center. Having full UPS and generator backup capacity for everything gets very expensive.

[technion]

I have multiple experiences with high end DCs with dual UPS and diesel genset experiencing power fail.

Once it involved fire alarms, which trigger safety shutdowns within a suite. The other involved a failed static switch panel - ie, the things that aren't mean to be able to fail.