[devnull42]

It appears that the issue at first impacted all servers in the anycast pool however eventually it only impacted servers ns-a2 and ns-a4. Those servers started returning NXDOMAINs. I am wondering if this was related to the root server key change yesterday. .IO seems to struggle with basic DNS engineering. We are seeing stabilization except for minor issues still on one of the gTLD servers.

[steventhedev]

The root server key won't actually change until next month. The DNSKEY responses from the root server was increased yesterday. Speculation, but it could be they're running ancient versions of BIND that fail with the larger response size. As per ICANN[0], the timeline is:

October 27, 2016: KSK rollover process begins as the new KSK is generated.

July 11, 2017: Publication of new KSK in DNS.

September 19, 2017: Size increase for DNSKEY response from root name servers.

October 11, 2017: New KSK begins to sign the root zone key set (the actual rollover event).

January 11, 2018: Revocation of old KSK.

March 22, 2018: Last day the old KSK appears in the root zone.

August 2018: Old key is deleted from equipment in both ICANN Key Management Facilities.

[0]: https://www.icann.org/resources/pages/ksk-rollover

PS - thank you for mentioning this, I wasn't aware it was going to happen until reading your comment.

[devnull42]

Correct the actual key change isn’t until next month however yesterday there was a change in response size from the root servers.