[bullen]

This is my experience of the outage: My DNS servers stopped working but HTTP was operational if I used the IP, so something is rotten with this report.

Lesson learned I will switch to AWS in Asia and only use GCP in central US, with GCP as backup in Asia and IONOS in central US.

Europe is a non-issue for hosting because it's where I live and services are plentiful.

I'm going to pay for a fixed IP on the fiber I can get that on and host the first DNS on my own hardware with lead-acid backup.

Enough of this external dependency crap!

[MayeulC]

> I'm going to pay for a fixed IP on the fiber

This is nice for backup, but I would expect more downtime from your ISP than the big cloud platforms. Also, you might want a platform with anycast DNS if you care about (initial page load) latency.

[bullen]

Sure you get more downtime, that's why I have 2x fibers with my 100% read uptime database between them, that way both fibers have to go down at the same time for existing customers to be unable to login.

I noticed DNS was a bit slow on first lookup, it's not a big deal for my product and well worth the extra control.

I looked up anycast, and it's unclear how you enable that if you have your own DNS servers, I have 3, one in each continent but I'm pretty sure the DNS provider I use does not use the DNS in the right region!

Is that something you tell the root DNS servers about through your domainname registrar?

You would think this had been built into the root servers ages ago? They can clearely see where my DNS servers are!?

[MayeulC]

> I noticed DNS was a bit slow on first lookup

Have you measured this from another continent? I noticed it could add quite a bit of latency, especially when the remote client has a relatively slow internet connection.

More specifically, I noticed that when I was using a CNAME to a domain with DNS in the US.

To use anycast, you need the same IP addresses in multiple locations. Realistically, you can only do that if you peer with local ISPs and can advertise a route.

I never dug enough to start my own ISP, so it's a bit fuzzy for me, but I think you need to control your own AS (or partner with one), and announce your routes over BGP from multiple areas.

Most CDN or cloud providers probably offer anycast as an option, and it is likely the default configuration for their DNS as well as static websites.

[bullen]

Aha, ok thx!

I'm going to add geolocation lookup on my own DNS eventually.

But my product will connect directly to each region and measure latency and the number of players so anycast would not help a great deal for the complexity.

I wish the DNS roundrobin used the order of the replies in the DNS packet as priority instead of randomly picking one IP... that way my DNS servers could direct people to the correct region without loosing the backup!

As to why the root servers are not doing geolocation lookups in 2021 I'm just baffled by the lazyness of monopoly owners, but then again the priority ordering would be needed first!

[breakingcups]

Anecdotally, I've had 100% uptime on my ISP for the past 3 years and have read many a cloud provider's post mortem in that time.

My company hosts a large portion co-located in a datacenter and has the same uptime as my ISP. Clouds seem to be more complex which invites more opportunity for things to go wrong.