There is work to give the user more control over this that unfortunately didn't make it into Firefox 4 [1], but ultimately it's a battle that can't be won; on the desktop, apps can modify other apps. I suspect that this will happen to Chrome too.
Heck, since Chrome has to put itself into the users AppData folder in order to do silent updates, is there anything preventing malware from just screwing with the Chrome libraries themselves?
Even though you can't uninstall it, you can disable the extension on a per-user basis. Firefox looks in both per-user and system-wide locations to find installed addons. Only per-user addons can be removed by the user, in the addons dialog. System-wide extensions can be disabled per-user, but not uninstalled (disabling is almost the same thing). To uninstall a system-wide extension, hopfully the publisher has an unistall script, but worst-case, you can just delete the extension folder from the system-wide location. e.g. C:\Users\All Users\Mozilla\something on Windows.
It's not a question of Firefox allowing this -- if you run any program on your computer it'll have access to your data (and hence to your Firefox profile), and one of the principles behind Firefox is extensibility.
This is a historical accident; only toy and mobile OSes have any sort of application-level access control.