[voidUpdate]

Is there a practical way to enumerate all the registered internet domains? EG by asking DNS servers for all the domains they know about, and repeating over all DNS servers they know about?

EDIT: apparently, "asking DNS servers for all the domains they know about" is not something you can really do anymore for security reasons. Guess that idea won't fly lol

[0x0]

Enumeration of the entire DNS space is not available in general, but it does appear that some TLDs offer complete zone files for legitimate research purposes, see for example https://czds.icann.org/help#zone-files

[ks2048]

Check out the "Root Zone Database",

https://www.iana.org/domains/root/db

(this doesn't have .yu)

[gucci-on-fleek]

> apparently, "asking DNS servers for all the domains they know about" is not something you can really do anymore for security reasons. Guess that idea won't fly lol

There are actually a few nameservers that will just give all their domains to anyone who asks [0], but they are very much in the minority.

[0]: https://github.com/acidvegas/mdaxfr#tlds-that-allow-axfr

[bit1993]

This https://www.whoisxmlapi.com/whois-database-download.php is a start, but there's new ones every second so you're going to need to update a lot.

[onion2k]

You could probably extract a lot from https://commoncrawl.org/

[ks2048]

Here's the latest (although, it looks truncated at those having > 1M pages),

https://commoncrawl.github.io/cc-crawl-statistics/plots/tld/...

[ccgreg]

The complete list hides in the web graph:

https://data.commoncrawl.org/projects/hyperlinkgraph/cc-main...

and the specific file that's every host we've seen in the latest 3 crawls is:

https://data.commoncrawl.org/projects/hyperlinkgraph/cc-main...

[basilikum]

certificate transparency logs will give you a lot of that data.