The walled garden I speak of is the implementation of add-ons cryptographically signed by Mozilla only. Unless you run one of the betas (w/bugs) you cannot use any add-on not approved by Mozilla. And lately they've even been removing/banning add-ons that aren't illegal but otherwise upset outside corporations (ie, the paywall bypasser that uses googlebot's http headers substituted in).
It's important not to lump together the proprietary browsers (where the limit you speak of can only be worked around if the proprietors allow) and free software browsers which respect your software freedom, like Firefox.
You could modify Firefox source code to let you run whatever add-on you wish. With Firefox whatever changes you want are limited only by your willingness to implement them (either by doing the work yourself or working with others to get those changes made).
You can even distribute the improved Firefox variant browser to others and help the community. Hacker News recently had a story about some hackers doing exactly this with a variant they call Librefox. Other examples of Firefox variants include IceCat and the Tor Browser.
Is there a browser you would recommend, if we ignored their HTML/CSS rendering for a second, based on ethical considerations, so to speak? I like making things compatible when I don't need anything modern, and would be interested in hearing about any browsers you think are worth supporting.