Hacker News new | ask | show | jobs
by dx034 1238 days ago
Shows that all these availability zones and regions don't really help if an outage can knock out a whole cloud provider. And that's not specific to Microsoft. The only way to really ensure uptime is to use two providers. Sadly, that's basically only possible with on-prem/colocation where traffic is cheap.
1 comments
sofixa 1238 days ago
It's mostly Azure though that is badly designed to such an extent that multiple times there have been global outages. In general Azure availability, security (the only major cloud provider with not one but multiple cross-tenant security exploits) and usability are pretty terrible so it shouldn't be used for anything but saying "this is how it should not be done".

GCP had a similar thing once, where a BGP update knocked out their Asian regions.

AWS have never had a global outage. (And no, that time S3 in us-east-1 was down wasn't a global outage, the only customer code/workloads that were impacted was code interacting with S3 that didn't specify the region and had to rely on us-east-1 to determine it, and it didn't work anymore)

link
Andys 1237 days ago
To be fair, AWS once had a global Route53 outage, which was effectively a global outage for anyone using AWS for DNS.
link
snorkel 1237 days ago
That outage was limited to Route 53 DNS record editing and not DNS lookups.
link
eurg 1237 days ago
Do you have a link to an article about that? My google-fu is weak, and this sounds interesting - that should not happen to DNS - at all - and from the outside Route53 looks quite well managed. So what the heck did they do?
link
codalan 1237 days ago
It was back in 2019.

https://twitter.com/AWSSupport/status/1186735657387003904

I forget the details. I do remember half of our internal tools not working at the time due to DNS issues, though. Good times.

link
wereallterrrist 1237 days ago
Someday someone will write a book about how AD, AAD, etc, exert the control they do at MS and go as unchecked (or at the time) as they do. AD's inability to execute made Azure a significantly less pleasant platform until they finally fixed accounts a couple of years back to properly do OAuth 2.0 with ARM.

Maybe the book is just "AD brings in the money" but wow, they sure bring it down as well. Global outages like that always stink of AD.

link
throwawaythekey 1237 days ago
There have been several cloudfront outages that have effectively been semi global outages
link