[xg15]

> And then people go "well you can statically analyze it better!", apparently not realizing that ESM doesn't actually change any of the JS semantics other than the import/export syntax, and that the import/export statements are equally analyzable as top-level require/module.exports.

...

"But in CommonJS you can use those elsewhere too, and that breaks static analyzers!", I hear you say. Well, yes, absolutely. But that is inherent in dynamic imports, which by the way, ESM also supports with its dynamic import() syntax. So it doesn't solve that either! Any static analyzer still needs to deal with the case of dynamic imports somehow - it's just rearranging deck chairs on the Titanic.

I think while OP's right in theory, there is still a lot of difference between the two: ESM has dedicated syntax for static loading of modules and that syntax is strongly communicated to be the standard solution to use if you want to load a module. Yes, dynamic imports exist but they are sort of an exotic feature that you would only use in special situations.

In contrast, CommonJS imports are dynamic by default and only happen to be statically analysable if you remember to write all your imports at the beginning of the module. That's a convention that's enforced through nothing and is not part of the language or even of CommonJS.

As an exercise, try to write a static analyser that simply ignores dynamic imports and just outputs a dependency graph of your static imports - and compare how well this works with CommonJS vs ESM.

[kabes]

It's not the fact that imports are top level that makes it statically analyzable. It's the fact that the imports can't be variables like commonjs allows.

[tolmasky]

But they can. I can construct any hard to find use case just as easily with import. Your analyzer probably won't find this: "(a => a)(path => import(path))(ROOT_PATH + "/x.js")".

"But no one would do that!" Yeah, and no one does it with require either. The times people pass in variables to require are well warranted and usually not in a browser context (listed this in my other post, but for example loading "x-mac.js" vs. "x-windows.js" in node, where there is no problem since you're not bundling).

So neither in the practical sense nor the hypthetical sense has the "static analysis problem" been solved any more than it was already solved with require. People used require basically like a static keyword before and you could basically make the same usage assumptions as import in those cases. Similarly, you can get as dynamic as require with import, so in the purely theoretical sense not much has been made better either.

[andrew_]

Say what now?

  const batman = await import('batcave' + version);

[kabes]

That's only valid for dynamic imports. But if we consider dynamic imports half of the rant in the post is wrong, since these can appear nested.

[tentacleuno]

> Yes, dynamic imports exist but they are sort of an exotic feature that you would only use in special situations.

I wouldn't necessarily describe dynamic imports as an exotic feature. They are basically required if you're building a semi-large app (and heavily advised by most frameworks in that case!). Otherwise, your homepage is going to load in some complicated AuthorPage / HeavyComponentWithLotsOfDependencies component the user might not even want.

The main advantage of ESM in this context would be its asynchrony, which you won't get with require. Webpack tried to tack this on with require.ensure, but it was nonstandard and eventually deprecated.

Another advantage of ESM in general would be that you wouldn't have to use preprocessors and compilers like Webpack. My main reasoning here is that, prior to the import spec, web didn't have any form of imports aside from loading in scripts and polluting the global namespace (which isn't necessarily bad). The majority of websites IME use some sort of bundler though, so this isn't really a major change so much as a nice to have, I suppose?

[crooked-v]

> that you would only use in special situations

There are web frameworks pushing this pretty hard for basic stuff (for example: loading React components with dynamic import) to build page-content-streaming functionality around it.

[tolmasky]

This isn’t a real concern, and yes I’ve written a static analyzer for requires (as have many build tools). The fact of the matter is no one is trying to trick the analyzer by passing variables to require, or even more mischievously trying to rename require or something (there aren’t a lot of “(a => a)(require)(path + “/x.js”)” out there).

In practice, it is used like a static feature, and when it isn’t, it’s for a good reason that import doesn’t solve and just expects you find a harder solution to. For example, if you want to load a platform specific file depending on your host environment. With import, the entire function now has to become needlessly async just because any use of the import expression needs to be async. Another good example is modules that put their requires inside the calling function to avoid needlessly increasing the startup time of an app for a feature it may not use. This way, only if you call that specific function will you have to suffer the require/parse/runtime hit for it. Notice all these cases are in node, so they wouldn’t result in some complicated decision as to whether to include these “dynamic requires” into the main bundle or not — it just doesn’t come up in bundling since they are use cases that are specific to node. But because of ESM, node now needs to make a bunch of synchronous functions be asynchronous to accommodate a set of restrictions designed with the browser in mind. And again, at the end of the day import does still have an expression form so you haven’t actually resolved the static analysis problem, just made dynamic imports more annoying in non-browser contexts.

[spankalee]

> to accommodate a set of restrictions designed with the browser in mind

That's the whole point, and a very good thing.

[tolmasky]

That was not the whole point, the whole point was to have a feature that could work in a variety of different environments. That’s why this language feature is in the ECMAScript spec and not in the W3C or whatwg, unlike something like “fetch” which is defined by the whatwg and thus has every right to not take other environments into consideration. There is a tremendous amount of subtlety that results from this fact, like how the spec can thus only define a small portion of this feature (syntax and basic semantics), but ultimately needing to leave everything related to fetching, resolving, and executing the code up to the environment (in HTML’s case, the whatwg HTML spec). This really complicates things and creates an unfortunate mismatch in expectations, where most users who have only a passing understanding of this feature and have been sold on the promise of something that will finally “just work” everywhere discover that this isn’t the case at all. There’s a reason why despite being introduced in ECMAScript over 5 years ago it still barely has support in node (and not great support in browsers either btw, but certainly better)— it’s because the reality is that this feature is supremely complicated to implement (despite providing very little tangible benefit), especially in the context of the unrealistic expectations users have developed for it as it continues to be pitched as being the magic tool that will make your code work everywhere without a build tool.

[spankalee]

It really is the whole point. TC39 was not going to define a feature that didn't work in browsers, full stop.

That Node has to take browser into consideration is a very good thing for universal JavaScript. We can now write code that works in browsers, Node and Deno and that's a great thing.

The support in browsers is excellent btw. All current browsers support standard JS modules now. Chrome is leading the way with import maps, import assertions, and JSON and CSS modules, but the other browsers will get there and CJS had nothing comparable to those anyway.

[tolmasky]

It is absolutely not excellent, unless you restrict yourself solely to whether there is a checkmark next to the browser name in mdn. It is very buggy, very difficult to debug, and as I mentioned in another comment, missing serious features (like no subresource integrity which, means we’re encouraging people to use a much less secure system of importing scripts in many cases!)

> It really is the whole point. TC39 was not going to define a feature that didn't work in browsers, full stop.

No one is saying they shouldn’t have considered the browser! We’re saying they should have also considered other major environments, like node! That’s the way to design a language feature and absolutely what they wanted to do. There were a number of reasons it was rushed out the door, but they’re pretty upfront about the fact that they would do things differently now and basically no other feature would be allowed into the spec in the state ESM made its way in then. I am currently a TC39 delegate and can assure you that it’s OK to admit when things aren’t great so that we can learn from it. It’s how we’ve gotten JS to such a better state than where we were 20 years ago, not by bending over backwards to defend the with() statement.

[spankalee]

I've been using almost exclusively standard modules for years and they work quite well. Old crashers and cache problems I was aware of have been fixed for many years now.

I don't know what debugging problems there are that wouldn't exist in CJS. At least with native modules you can see individual requests in the network panel while you're working and individual files in the sources panel while debugging. That alone is a huge increase in debuggability to me.

SRI really should be done out of band. Inline SRI would require far to frequent cache invalidation and isn't compatible with package manager workflows where you don't know the exact version of a file you'll depend on. A tool rather should build up an SRI manifest similar to a package lock. This has been discussed several times in module and import map threads.

And I don't think JS modules were actually rushed. They were languishing for years with an overly complex loader API and cut down to an MVP with the core agreed upon semantics and that's what finally got everyone to ship. They're really fine, and with import maps, CSS modules, and eventually web bundles, will be far, far superior to any previous alternative. They already are.

The fact that Node has some JS module / CJS interop issues (and really only when you try to use JS modules from CJS, the other way is fine) is Node's problem, not TC39's. There's really nothing that TC39 could have done here because synchronous require() is the fundamental problem. It was a bad design from the start and we shouldn't burden ourselves with that bad decision forever. The sooner CJS goes away the better.

[wruza]

How browserify, webpack transformers and others are able to parse require()-s in the middle of a source file, but static analysers are not?

These subtly erroneous arguments are the essence of this push. Look, we are maintaining X, Y and Z, and they’re unable to do that R, so it’s bad. No, it’s you making them unable to do that consciously.

[nosianu]

1) importing module

"require()" can take a non-static string. A variable or a string calculated at runtime.

You can only statically check if that particular feature is not used, but there is no checking the entirety of what require() can be used for/with.

require() is more like dynamic imports in Es modules that are awaited and not like the static ES modules.

2) exporting module

The other issue is on the exporting module's side: You can do strange things with the "exports" object. ES module exporting is more strict to make it guaranteed statically analyzable.

[wruza]

Es modules that are awaited and not like the static ES modules.

But this is a non-argument. Awaited or not, you can’t statically check them either. Developers aren’t idiots and they can read “if you pass a string to require/import, tooling can’t figure that out” in a manual.

You can statically check “require(literal)” and cannot “require(variable)”. You can statically check “import from literal” and cannot “import(variable)”.

And awaited imports are essentially just memoize(name => (fs.readFile || fetch)(name).then(wrapAndEval)))(name). It’s not a black magic that is available only to “imports”.

do strange things with the "exports" object

It’s just a value. Somehow analyzers/checkers can work with “var x = do_strange_things()”, but can’t with exports. How is a module boundary different from any other expression boundary?

I really don’t want to think that this is pure zealous smoke blowing, but these arguments are as weak as nil, and leave no options.

[jokethrowaway]

You can detect from the AST whether a require can be statically resolved or depends on non static variables.

It may not be as simple, but I venture it's easier to implement than having all the software ever written needing to be migrated

[xg15]

You can detect simple cases like require("some string literal") on top-level. Things become harder if you have require() in init functions or wrapped in (function(){})() or if the module name is defined in a string constant elsewhere.

I can't say how common those forms are, but my point is that there is nothing immediately discouraging a programmer from using them as all of it is "just javascript". So even if you just restrict yourself to static imports, its hard to be sure you caught all of them without running the program.

[wruza]

Wrong, things do not become harder when require() is nested somewhere. It may not get called in node, but any bundler looks at it as required-to-be-bundled-anyway. The only case when it’s hard is when require() accepts a non-literal, and that’s symmetric to import(). No extra cases.

The programmer is discouraged to use dynamic requires/imports by the common sense, because that makes their app/lib incompatible with most of the cross-end usage. But then we have server-side only packages like pg or express where it doesn’t matter, because browsers provide no runtime (tcp/listen) for them to function and will never do.

[forty]

Unless I'm missing something, the exercise is pretty simple, and could be done with grep in a sane codebase (find all require without indentation and with a single string literal inside the parenthesis)

[joepie91_]

It's a bit more complicated than that because you'd not want to match such things if eg. they exist inside of another string literal; but the actual implementation of 'dependency collection' in various bundlers isn't too far off from this. They just often do an AST match instead of a string match.

[brundolf]

> Yes, dynamic imports exist but they are sort of an exotic feature that you would only use in special situations.

They're also decidedly inconvenient to use in most situations, because they return a Promise (unlike require()), which means anything that depends on them has to itself be deferred. This further discourages using them unless you really need to.

[agumonkey]

now I see files as syntactic module level lambdas where args comes first and the body is the rest :)