I'm having trouble understanding your example. Who is the correct owner of dev.app.org.dept.nsw.gov.au and how does Let's Encrypt get it without the use of such list?
Each and every level of that hierarchy has a different owner! That's why the domain name system is a hierarchy and not a flat list, to enable this delegation capability.
In this case, it was something like:
dev.app.org.dept.nsw.gov.au -- Outsourcer (private company)
app.org.dept.nsw.gov.au -- Project Team in "org" managing the outsourcer
org.dept.nsw.gov.au -- A small government agency
dept.nsw.gov.au -- A huge super-department that manages the agency
nsw.gov.au -- A *different* super-department
gov.au -- The federal government
au -- The federal government
Each level is a separate set of name servers, DNS zones, and hosting platforms. Each group "owns" and manages DNS zones at their level. Not some arbitrary, random level above them based on some random text file some dude at Mozilla maintains for GUIs and cookies.
At the leaf level of that heirarchy I was trying to deploy Azure "ARM templates", which are supposed to be fully automated. My account permissions were scoped to the Resource Groups and existing resources associated with the non-production application environment. I could create DNS records, for example, but only under "dev.app.org.dept.nsw.gov.au".
Instead, the Azure platform insisted on trying to email "admin@nsw.gov.au", which is a totally unrelated department. They rightly would see this is a phishing attempt, and even if I were to call them on the phone their right move is to reject the request. Or even call the police. The best option here is to get people from my org's department to pass things up the chain, then across to the dept managing "nsw.gov.au", then down to some tech. This takes about a month.
What the Azure platform should have done is requested that I create a marker record such as a TXT record such as "_verify.dev.app.org.dept.nsw.gov.au" or whatever. This can be fully automated as a part of a scripted or templated deployment. Emails to random bureaucrats... not so much.
I guess that, regardless of this, there is still an ownership difference between the multiple levels in your example, and the ownership difference between a TLD and a SLD. In your example, nsw.gov.au controls the TLD and they could register dept2.nsw.gov.au without asking anyone else, they could even register a.b.c.d.dept2.nsw.gov.au, but they can't register nsw2.gov.au. I assume that's why the PSL is still useful for some use cases, but I agree this one is definitely not one.
With Let's Encrypt your script can just publish a dns record "_acme-challenge.dev.app.org.dept.nsw.gov.au", and Let's Encrypt will verify it based on DNS delegations. The fact that you can publish it means you control/own the domain. A similar thing occurs implicitly with HTTP verification, the A record verifies that the owner of the domain trusts the web server (in some sense).
It sounds like Azure require some kind of manual, out of band verification. Maybe they tried emailing a well-known email (like postmaster@_nsw.gov.au), based on information from the PSL. A tiny contractor deploying a single application may control that one URL, but not the whole domain.
Letsencrypt still has a notion of a domain against which all the quotas are computed. If they incorrectly determine the domain an operator could have a configuration mistake and deplete the quota of an unrelated domain
Similarly, Azure can't create certificates correctly for agencies or departments in the Australian Capital Territory (act.gov.au) and Northern Territory (nt.gov.au).
In this case, it was something like:
Each level is a separate set of name servers, DNS zones, and hosting platforms. Each group "owns" and manages DNS zones at their level. Not some arbitrary, random level above them based on some random text file some dude at Mozilla maintains for GUIs and cookies.At the leaf level of that heirarchy I was trying to deploy Azure "ARM templates", which are supposed to be fully automated. My account permissions were scoped to the Resource Groups and existing resources associated with the non-production application environment. I could create DNS records, for example, but only under "dev.app.org.dept.nsw.gov.au".
Instead, the Azure platform insisted on trying to email "admin@nsw.gov.au", which is a totally unrelated department. They rightly would see this is a phishing attempt, and even if I were to call them on the phone their right move is to reject the request. Or even call the police. The best option here is to get people from my org's department to pass things up the chain, then across to the dept managing "nsw.gov.au", then down to some tech. This takes about a month.
What the Azure platform should have done is requested that I create a marker record such as a TXT record such as "_verify.dev.app.org.dept.nsw.gov.au" or whatever. This can be fully automated as a part of a scripted or templated deployment. Emails to random bureaucrats... not so much.