The question is why are these things even limited to a fixed predefined set and not allowed to be arbitrary of some max string length, given that the rest of the domain name already is.
Or why even require a fixed extension given that it's a recursive system anyway.
There's a number of complications with a wide open gTLD system.
1. Local DNS resolution. Bare (non-gTLD) DNS names are common in business networks, and although the expansion of the existing gTLD set complicates that, opening it wide can have unintended resolution impacts for these networks.
2. Browsers enforce security boundaries based on the origin of a site, and doing this is complex due to things like .co.uk. A naive implementation would grant all .co.uk domains into a single origin. The problem comes into play when looking at local DNS names. Without a known gTLD on the end of the DNS name, the browser assumes the root of the origin is the hostname itself. That is, given a locally resolvable http://bar and a locally resolvable http://foo.bar, both sites can communicate without restriction with respect to the Same Origin Policy. If we were to make gTLDs arbitrary, we'd have to break that behavior which could break a lot of intranet applications.
Or why even require a fixed extension given that it's a recursive system anyway.