Unprocessed ESM in the browser makes sense for local development. We are collectively wasting millions of CPU hours (and developer time) waiting for our mostly unchanging dependencies to be processed. For production deployment though, I'd still prefer ESM in the browser, but not verbatim as they are coming from npm, but compiled, minified, and bundled in a way that strikes a balance between total number of modules, code duplication inside the modules, long-term cacheability etc.
Just as an aside... every web app I write these days starts with an index.html page that has a window["deploy"] bool at the top. If that's false, the first script just requires the unbuilt files. If true, it requires the compiled and minified version. I only rebuild when I'm ready to upload.
The more people start to internalize the truth that "if you ship it you own it" and stop adding dependencies and start removing them, especially if they come with their own wasteful dependencies, then ESM will make sense for everyone. Until then, you're right, devs have to go through unctuous mitigations.
It would be easier to do that if proposals for things like standard library functionality (not the contents of the standard library itself, just the syntax and technicalities of using it) were to go anywhere in, say, under five years.