[deathanatos]

> and use short-TTL weighted CNAMEs, defaulting to say, 99% Cloudflare, 1% your internal load balancer. The minute Cloudflare seems problematic, make it 0% 100% to bypass Cloudflare’s infrastructure completely.

Except if you're using CF for DNS service, this wouldn't have worked, as both CF's website & DNS servers were impacted by the outage.

[txcwpalpha]

That can't be possible, CF's website explicitly says that their DNS is "always available" with "unparalleled redundancy and 100% uptime"! ;)

In all seriousness, I wonder if they are going to have to change the marketing on the site now...

https://www.cloudflare.com/dns/

[hunter2_]

Somewhere, someone is saying the marketing is fine since 100 has just 1 significant figure and therefore two nines -- no, one and a half nines -- can safely be written as 100.

[esperent]

If they are truthful it will now probably change to 99.997% uptime or similar. I expect that's still good compared to many DNS providers.

[belorn]

Some dns provider only have one name server which is naturally bad.

However it is a bit bad of CF that a single configuration error can bring all the slave servers down. It mean that they have no redundancy in term of BGP misstakes. Customers of CF that want to avoid this would benefit to add an additional slave server outside the hand of CF.

Zonemaster (dns sanity checking tool) actually complain about CF hosted domain names because of the lack of AS redundancy. The outage yesterday demonstrate nicely why that is an concern and why one should care. https://zonemaster.iis.se/?resultid=7d1fab165987e195

[jontro]

Same goes for route53 too unfortunately

[QuentinM]

Yeap, that's specifically what I also implicitly meant by "Spread your name servers" (besides having them distributed). To use this technique, you also must have a "Business" account with Cloudflare ($200/mo), so to leverage their ability to front your websites using CNAMEs : )

[deathanatos]

Ah, I see.

This got me Googling, and best as I can tell, CF doesn't support zone transfers. (They support being a client, but not a server. So, they could function as one's secondary system, but not as the primary.)

[stedaniels]

I imagine/hope a lot of the heavily engineered ops teams don’t use zone transfers, instead using APIs. The idea of primary and secondary wouldn’t really a strong indicator at that scale.

[deathanatos]

Sure, I could use the CF APIs… but to do what? AFAICT with a quick look over the documentation, there isn't any way to tail a log of changes being made to the zone. (You can export the entire thing in BIND format, though, the example in the docs has several errors in it that make me wonder how well that would work.) (The idea with zone xfers is that it is at least semi-standardized, whereas CF's API, while useful, is not.)

Then I'm stuck with a bunch of bad questions about how often to poll, and whether CF's rate limits would support any reasonably quick poll interval.

(The big problem is that we have other tooling that relies on being able to update DNS, the big one being ACME for certificate renewal. The changes it makes to CF would need to be rapidly replicated out to the nameserver.)

(Nothing in the post really strikes me as particular to CF, either. I think I could easily replace everything I've said here with "Route 53" and end up in the same bucket, maybe plus or minus zone xfers working.)

[foobarbazetc]

Using CF for DNS is, IMHO, a bad idea in general, especially for large sites.

We use AWS + Azure + GCP (yes, all 3) as our authoritative NS and keep them all in sync with octodns.

[neillyons]

Ah ha, This probably explains why my internet stopped working for a bit. I had the 1.1.1.1 app from Cloudflare installed on my phone.