[torginus]

> But our VMs start fast, so fast that even if we created the DNS entries before creating the VM, our users still had to sit around waiting for DNS to propagate, which occasionally took minutes, not seconds.

To my (limited) understanding, this is not a good idea, and is an unfixable problem from the server side. Companies, VPNs or ISPs or routers, often use their own DNS servers, and those can have caching logic, which means it doesn't matter how fast your own DNS implementation is, as the users lookup request wont hit your DNS server, it'll hit an intermediate cache.

[tibordp]

Presumably these are new subdomains, if a caching DNS resolvers never saw the domain yet, it would simply query the authoritative server.

There's a small nuance here that DNS spec also caches negative (NXDOMAIN) results, but the TTL of those is controllable by the SOA record on the domain that contains it.

exe.xyz has a fairly short 10 second negative TTL, so even if you tried to resolve it before it was set up in the authoritative DNS, it wouldn't stay negative-cached very long (though some DNS caches put a lower bound on the TTL and cache it for longer)

> dig +short soa exe.xyz

ns1.exe.dev. hostmaster.exe.dev. 1854440 86400 7200 1209600 10

[eigencoder]

Usually when you create the VM, it gets its own DNS subdomain that looks like `my-new-vm-name.exe.xyz`. (I'm not affiliated with exe.dev, I just use their product). So I doubt there would be any cached requests for that particular subdomain before the VM is created.

[dspillett]

NXDOMAIN results and other errors have their own TTL value though, so if someone queries too early in the process you could be waiting a little while for their cache to check again. The value for this is set in the SOA record for your DNS zone.

For example, example.org/example.com seems to have TTLs set at 300s (5 minutes) for A records, but the negative caching value is 1800s (30 minutes). If your VM domain is set the same way, you could see a premature lookup causing issues for 30 minutes instead of just 5. Even the 5 minutes could be a pain if your VM spin-up process expects to be able to lookup via the name very early and will fall over if it can't.

[eigencoder]

I always don't know enough about DNS. This is great stuff.

If exe.dev is running their own DNS zone, then wouldn't they be setting the expiry value in the SOA record themselves? So they could set it to something short, and secondary servers are supposed to honor it, right?

[dspillett]

> wouldn't they be setting the expiry value in the SOA record themselves?

If they thought about it. But it is something that even people relatively experienced in DNS matters forget about until it bites them and they've spent ages diagnosing the initially odd looking problems caused.

[indigodaddy]

Looks like the NXDOMAIN TTL is 10 seconds:

https://dns-lookup.jvns.ca/#exe.xyz|SOA

[dspillett]

> To my (limited) understanding, this is not a good idea, and is an unfixable problem from the server side. Companies, VPNs or ISPs or routers, often use their own DNS servers, and those can have caching logic, which means it doesn't matter how fast your own DNS implementation is, as the users lookup request wont hit your DNS server, it'll hit an intermediate cache.

All caches should respect the TTL of DNS records, no matter what they are, though there are some interesting bugs out there.

Firstly the human bugs: a lot of people assume switching DNS is always fast, but they haven't had the TTLs property of a DNS entry explained to them so assume that if the change is working here then it is going to be working <there>, <there>, and <everywhere-else> too. The default TTL values are a mix, it used to be that 24 hours was common though these days four hours or less, sometimes even one hour, is what you are more likely to see even for defaults. Whatever value your entries have though, that doesn't mean a fixed × hour window that it will switch at the end of. If the TTL is four hours your local cache last checked three hours ago but mine last checked one hour ago, a change will take an hour for you to see and three for me to see. A cache that has not recently looked up the entry will see it immediately next time it is asked - this is why some assume it is always instant (they add a new sub-domain record and it works instantly because no one has ever requested an address for that name before, for instance, and assume this is how it will always work).

Then there are coded bugs. TBH these days I ignore those: they are rare enough that if you are caring about that sort of edge case a lot then there are a lot of other weeds you'll be worrying about too and you'll never get anything done. For my domains I mostly have everything set to five minutes (300 seconds) as they are rarely referenced and DNS infrastructure is not a high-power service these days. Some would suggest that using such a low default is unfriendly to DNS caches and the root servers, but my names are very rarely referenced outside my network/vpn where my own DNS authoritative servers are what get directly queried anyway so that isn't going to matter. example.{org|com} and some sizable companies use 300s too. Why five minutes and not less? There used to be a very common caching DNS server that would ignore anything lower than 300 seconds and apply its own default (14,400 seconds, four hours, IIRC) instead. This goes back a couple of decades though, if anyone is still running that they deserve some DNS lookup failures! If you are wanting to use longer values normally, just remember to lower your TTL a time before any planned change so the change propagates ASAP and raise them back up after. One extra thing to be aware of with short TTLs is an outage affecting all your authoritative DNS servers may cause less friendly errors in many cases than getting an out of date address would result in, so make sure your DNS servers are stable and have few (preferably zero) shared potential points of failure.