[teilo]

Elastic Load Balancer, anyone? I have no sympathy for this company. They deserve to get sued for this.

Yes, for something life threatening like this, EC2 is a bad idea, but they didn't even bother to take advantage of the geographic redundancy, much less something so basic as having backup AMIs ready in another AZ in your current region.

Of course, a whole lot of companies are learning this now.

[kinofcain]

ELB doesn't work across regions, the failure at EC2 affected multiple Availability zones in the Virginia Region. ELB might have helped, but it's likely that some people had all their "availabiliy" zones knocked out.

In fact, it's looking like all the automatic-failovers from people hosting on those data centers compounded the problem when everyone's scripts tried to recover at the same time – what was referred to in a previous article as a "bank run".

But yeah, if you're doing heart monitoring, you need to have your servers replicated across a wide geographic area.

[gregburek]

Unfortunately, this last AWS problem affected multiple AZs in the us-east region. The OP very well may have had an alternate AZ failover plan, but like Quora, Sencha, Reddit, FourSquare and Heroku, they probably kept it region specific.

As for backing up to multiple regions, I can imagine them thinking that sending everything over the public internet as being a bad idea. However, not having a multi-region/multi-provider failover plan was a worse one.