[sirius87]

The current ESM experience makes it seem like decision-makers in the node.js project were wilfully oblivious to how large the TypeScript community was and how it was being used in node modules and projects. It really does feel like the maintainers focus was on JS and harmony with TypeScript's evolution was low on the priority list.

Meanwhile anyone using an intersection of TypeScript with jest and any of sindresorhus' libraries when he flipped to ESM for his bajillion libraries immediately felt the downside and moved hard away from ESM.

Imagine the mind-boggling hours lost just to get these export/import formats to glue.

[madeofpalk]

I really enjoy frontend/node/typescript development. I roll my eyes whenever the HN-types complain about CSS or frontend development being a hellhole. Mostly the comments I see seem ignorant or impatient ("Why doesn't this thing work without be bothering to learn it?")

However, the intersection of typescript, nodejs, and ES modules is consistently the most frustrating experience I ever have. Trying to figure out which magic incantation of tsconfig/esbuild/tsc/node options will let me just write code and run it is a fools errand. You might figure something out, and then you try to use Jest and then you descend into madness again.

The biggest tip I can give people is to ditch ts-node and just use (the awkwardly named) tsx https://github.com/privatenumber/tsx, which pretty much just "mostly works" for running Typescript during dev for node.

The problem mostly seems to stem for all the stakeholders being pretty dogmatic to whatever their goals are, rather than the pragmatic option of just meeting people where they are. I really wish the Node, Typescript, Deno/Bun, and maybe some bundler people would come together and figure out how to make this easier for people.

[johnfn]

Agreed with everything, except this:

> I really wish the Node, Typescript, Deno/Bun, and maybe some bundler people would come together and figure out how to make this easier for people.

Bun has solved this. Bun is straight-up magic; they've implemented tons of hacks and heuristics so everything just works. Bun can even handle ridiculous or otherwise invalid code, like having import and require in the same file.

[AntonCTO]

Unfortunately, bun is unusable due to a myriad of bugs. I closely monitor every bun release, hoping it will function well beyond simple node use cases. The idea is amazing, and I would love to switch to bun, but looking at the issues - no, not yet. How can I trust bun to be a secure runtime with all these bugs?

[johnfn]

Which bugs are you referring to? Every runtime has bugs in the issue tracker; you'll have to be more specific than that.

[AntonCTO]

I ran into all types of bugs, including segfaults and incompatibilities with node. After encountering bundler bugs like https://github.com/oven-sh/bun/issues/6168 and https://github.com/oven-sh/bun/issues/4216, I stopped finding workarounds to get bun working.

[madeofpalk]

My point here is that Bun should go teach Node how to be pragmatic and build a good developer experience

[johnfn]

Oh, yeah, totally agreed then.

[yencabulator]

> tons of hacks and heuristics

So, business as usual in the Javascript space. That's how the ecosystem got here, and why nothing works reliably.

[johnfn]

Like I said in my comment, Bun works just fine. You seem to have taken 5 words I said out of context to justify something that I didn't say. It's not true that "nothing works reliably". Bun works reliably.

[edgyquant]

I’ve never heard of bun before today. Why is it so low key and what’s it main selling point?

[flockonus]

let me HN it for you - https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=tru...

[johnfn]

It's a pretty recent development in the frontend landscape. Its main selling point is a fabulous developer experience. That and, like I said, combing over the ESM mess and basically Just Working (TM).

[kristiandupont]

What are your main sources of tech news? Bun has been quite a darling here for a while..

[phatskat]

My own primary news source these days is HN and I’ve missed anything about it, your comment isn’t really helpful.

[mardef]

Bun has solved this, unless you have developers on Windows.

[kaliqt]

How hard would it be to migrate a node project to Bun though? I want to move, but I'm not sure.

[AntonCTO]

TL;DR: It is too buggy.

I tried it and ran into many issues. For example:

- Incompatibility of transpiled artifacts with the Node runtime (many bugs were fixed, but I haven't tried again). I don't trust bun as a runtime.

- Broken monorepo support.

- No direct react support; you still need react-scripts/webpack/whatever. They have documentation on transpiling react, but it isn't the same.

- No type checking, as with many other tools.

Also, filter their issue tracker for bugs: https://github.com/oven-sh/bun/issues?q=is%3Aissue+is%3Aopen...

[johnfn]

Should be a drop-in replacement.

[sirius87]

Totally agree.

It's doubly frustrating because a standard for authoring modules across browser and server platforms such as ESM is a good thing. But it's a bit arrogant to expect module authors across TS and JS ecosystems to ship overnight. Beginners may just turn to Deno or Bun simply because hundreds of coding tutorials and snippets no longer work.

Or, when you finally get a TS config that works but then you import @aws-sdk/* or prisma seeds and then you really rip your hair out.

[quickthrower2]

I am finding whatever NextJS does “just works” and I hit issues when I roll my own stuff. But I have’t dug deeply into how they do it yet.

[qudat]

> I roll my eyes whenever the HN-types complain about CSS or frontend development being a hellhole. Mostly the comments I see seem ignorant or impatient ("Why doesn't this thing work without be bothering to learn it?")

I’d also argue that outsiders looking into all the complexity are ignoring the complexity within their own specialization: https://bower.sh/front-end-complexity

It’s hypocritical thinking.

[Shacklz]

Couldn't agree more. Love the frontend-space, love the ecosystem, but hate the whole ESM vs. CJS fiasco with a passion.

To some degree, I think the typescript-team itself also has to take some blame here. I understand their point that they do not want to do any rewrites, and to some degree it makes sense, but if the ecosystem as a whole really wants to move forward to a common understanding of how it should work, someone needs to do the heavy lifting for dev-experience, and right now they're best-equipped to actually solve the problem, or at the very least help us a lot in doing so.

Their dogmatic approach makes sense for the scope they set out with when starting with typescript but in my eyes refuses a bit the reality the ecosystem currently finds itself in. And I'm saying this as an absolute ts-fanboy; it's one of the very few things about typescript that I take an issue with.

[eyelidlessness]

I’m curious what it is you think TypeScript could do, or could do differently, to address the situation. Or what you think they’ve been dogmatic about, and what reality they’ve refused [to see? to accept? to fix? unclear what you mean]?

[cjr]

Hard agree, especially with your first two paragraphs :)

The biggest tip I would add is to bin jest and start using vitest.

[brundolf]

For projects where I can, I've moved to Deno and never looked back. It's blissful over there

[yencabulator]

Except when you need libraries from npm, and then you have a choice 1) use npm: imports and watch things not work 2) use esm.sh imports and watch supposedly-immutable URLs change their contents all the time, and be ready to pile on kludges to get transitive dependencies to behave.

[bartlomieju]

Bartek from the Deno team here. You can also use `package.json` and bare specifiers with Deno. We also recently added `--unstable-byonm` flag (Bring Your Own Node Modules) that allows you to manage `node_modules/` be the package manager of your choice. In other words, you should be able to drop Deno into an existing Node project.

[brundolf]

I have had mixed results when using NPM libraries, but I also haven't run into any cases yet where I absolutely needed them

Some stuff you might reach out to NPM for is built into Deno's standard library. Other stuff has become built-in JS language features over the years. And the ecosystem, while nowhere near as big as NPM's, seems to have all the most important stuff at this point, at least for back-end work

And if I have to fill gaps by reinventing some small wheels here and there, or by interning and converting an NPM library, that's still worth it to me to avoid all the tooling headaches

[zerobrainwash]

Relieved to hear I’m not the only one. I always blamed myself for not understanding it deeply enough. But admittedly, it is a shit snow and the most frustrating part of development.

[lacrosse_tannin]

How about not using Jest?

[teaearlgraycold]

Once Bun is feature complete I’m jumping ship

[galactus]

So much this.

[mofle]

TypeScript had years to prepare for ESM, but they did not. Same with Jest. ESM was developed in the open and anyone could participate, including the TypeScript team. You are talking like ESM just happened overnight. It had been in development for 10 years.

Node.js released initial ESM support [1] in Node.js 12.17 in May 2020, 2 years later (!), TypeScript finally added support for ESM [2].

Here's a straight forward guide on how to use TypeScript with ESM: https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3...

[1]: https://nodejs.org/en/blog/release/v12.17.0

[2]: https://devblogs.microsoft.com/typescript/announcing-typescr...

[wwwigham]

> ESM was developed in the open and anyone could participate, including the TypeScript team.

This point stings for me, personally, since _I_ was the TypeScript language dev _in_ this wg trying to make our concerns noted, because we certainly did participate. However the group largely deadlocked on shipping with ecosystem compatibility measures, and what you see in node today is the "minimal core" the group could "agree" on (or be bullied into by group politic - this was getting shipped weather we liked it or not, as the last holdouts). The group was dissolved shortly after shipping it, making said "minimal core" the whole thing (which was the stated goal of some engineers who have since ascended to node maintainer status and are now the primary module system maintainers), with the concerns about existing ecosystem interoperability brought up left almost completely unaddressed. It's been a massive "I told yo so" moment (since a concern with shipping the "minimal core" was that they would never be addressed), but it's not like that helps anyone.

Like this shipped, because _in theory_, it'd be a non-breaking from a library author perspective to get node's CJS to behave reasonably with ESM (...like it does in `bun`, or any one of the bundler-like environments available like `tsx` or `webpack` or `esbuild`), and _in theory_ they're open to a PR for a fix... I wish anyone who tries good luck in getting such a change merged.

[stephen]

Fwiw I appreciate your effort! That sounds really frustrating.

I agree the recent bun/tsx/esbuild (but bun especially) has shown the node CJS/ESM fiasco was a bit of an emperor-wearing-no-clothes moment, where I think us every-day JS programmers just trusted the node devs at their word, that CJS/ESM had to be this painful...

But now, seeing that it doesn't have to be that way, it's like wait a sec...the last ~5 years of pain could have been avoided with some more pragmatic choices? Oof.

[eyelidlessness]

> TypeScript had years to prepare for ESM, but they did not.

How did they not? This is confusing to me, as I’ve been authoring and emitting ESM in TS for quite a few years now.

[sirius87]

Curious to see how this one decision plays out for nodejs IF -

- module authors increasingly adopt ESM

- TypeScript experience with Deno is butter smooth

- npm packages work with Deno

- performance characteristics are similar

- DX for beginners just works (compared to thousands of stale articles with `require('pkg')` and `npm install pkg@latest` for nodejs)

[joshkel]

As someone who uses Jest and alternatively enjoys and is frustrated by it, this isn't all Jest's fault. Node is just now releasing fixes for [significant memory leaks](https://github.com/jestjs/jest/issues/11956#issuecomment-180...) around ESM support within the [VM APIs](https://nodejs.org/api/vm.html) that Jest depends on.

[unflxw]

My impression is that Node.js has to be dragged kicking and screaming into any modern JavaScript development practices, and each time, they try to support as little of it as possible as they can get away with.

They have only last year actually rolled out a release that uses ESM modules by default, when the bulk of the community has adopted TypeScript and moved on to using ESM imports (even if they are importing CJS modules under the hood)

They were really late to async/await, and even more so to promises. The chaotic situation where each dependency bundled its own promise library remained in Node.js years after browsers shipped built-in promises. And it still hasn't trickled down to their standard library, most of which is still callback-based and needs to be "promisified" manually. Even their support for fetch, which is eight years old at this point, is still marked as experimental.

[rtpg]

Honestly I think the blame[0] is to share across the board. I have read through a bunch of typescript lang issues on this topic, and the general issue is that Typescript doesn't want tsc to do certain kinds of rewrites during compilation (code transpilations that go beyond the most simple things).

This makes sense but honestly means a lot of QoL fixes are kinda impossible to do. At this point I, personally, would almost want tsc (the compiler component) to stop being almost a good bundler and remove some features. Every project I've worked on that uses tsc directly ends up needing a "come to Jesus" moment where a "real" bundler gets introduced and suddenly a lot of stuff becomes easier. Doubly frustrating for me to feel this because tsc does a lot of stuff and it is hard to imagine how Typescript the language moves forward without Typescipt the compiler. I just really don't want to type `.js` for files that are, pre-build, `.ts`.

[0]: "blame" here meant in the weakest sense of "there are people who could make decisions in another way to make this better". Not so much in assigning moral blame

[iainmerrick]

I still don’t fully understand why TypeScript needed to be involved in the discussion at all. TSC tries to do far too much and doesn’t do most of it very well. The tsconfig format is restrictive and annoying.

TS would be a lot better if it were just a type checker and nothing else (which is more or less what you get when you switch to a proper bundler).

[rtpg]

I think it's tricky, because if Typescript _didn't_ show up with tsc, then they would have had their fate bound to Webpack or something like that. And of course tsc and the language server go hand in hand. It's hard to understate the positive effects of tsc existing across the entire ecosystem!

I suppose in a way I'm complaining about a (mostly) non-issue, since bundlers all have good typescript support at this point.

[iainmerrick]

Yeah, overall I love TS as a language, and the tooling can be made to work well.

I just feel like there's a bunch of functionality in TSC that's more troublesome than it is useful. The same could be said of Node, of course.

[alexcroox]

Experienced that this week! Upgraded to node 20 and typescript and his p-limit lib was the source of a lot of pain. No matter how much I changed the tsconfig I just couldn’t get it to work with the subpath imports his lib used. Ended up downgrading the lib to previous major and moving on.

[andrewaylett]

The good news is that one may use `expect` with Node's built-in test runner -- the result feels fairly similar to using expect with Jest.

Indeed, Node's test support can handle dynamic test creation, so one can do crazy things like https://github.com/andrewaylett/prepackage-checks/blob/main/... -- that dynamically asynchronously executes NPM builds from subdirectories, loading and running per-build expectations.

[wruza]

when he flipped to ESM for his bajillion libraries

But that was just a short migration period. It's only been three years and we almost fixed all these issues. This one is probably the last one, this month.

[sirius87]

FWIW it wasn't meant to sound critical of him, his contributions are much loved and his attempt to push to ESM was also in good spirit with his helpful gist on the issues, but ESM was no silver bullet for the wide mix of imported modules in some projects.

I remember spending a weekend simply inlining all his module code directly into a project to sidestep the type module change. Fun.

[wruza]

Not fun really. I am critical (of everyone involved). We were asked to migrate under a promise it'll be over soon. "It only needs a little kick", they said, ignoring everyone and vandalizing the loved contributions. Years passed and we:

- still stumble upon it even in fresh projects

- have no esm-only bundle-less projects in practice, because it's still impractical

- learned a bookshelf each about the thing that should just work

- watch entire toolchains decay into a shitton of date-dependent issues

- have no clear troubleshooting paths, even when we have a clue what's wrong

They forcefully promised some bright future, we helplessly agreed.

Where the bright?

Three years later, I lost half an evening to [1] again. My love and politeness reserves are depleted.

[1] https://news.ycombinator.com/item?id=38365983

[w3news]

That it why i use native functions. No typescript syntax, no jest. For testing i use just the native test runner, and that works great with ESM. Jest has indeed still not good ESM support, you can do some Babel trics, but makes the process too complex. Best is to find an alternative, like the native test runner. Also typescript try some things that are not stable at ecma. It was too soon with the import/export, and dont use the native way. Also commonjs is still the default at typescript after compiling, but Node is moving to ESM as a default. I hope typescript will use more native functions that are already available and dont use their own way.

For me typescript syntax and jest is a no go.

Typescript is useful to check types, but i write just JavaScript with JSDoc, and check it with eslint in typescript modus.

[jampekka]

Makers of ESM were wilfully oblivious to sanity. CommonJS was/is one of the, if not the, best module and dependency systems out there.

https://gist.github.com/joepie91/bca2fda868c1e8b2c2caf76af7d...

[madeofpalk]

I find this mostly unconvincing (except for the part where all the tools messed this up and made the problem more difficult).

It's not clear to me how you get top-level await with CJS with significant drawbacks OR breaking changes that would just end up back up in ESM land.

[jampekka]

The convoluted byzantine mess that is ES top level await is a self-inflicted problem stemming from the ESM bizarre pipe dream that static import won't be a blocking operation for code that depends on the imported code.

With blocking require there's of course no problem with top level await. And having a thenable import doesn't even need any language level support.

Yeah, you can parallelize static imports. But you can just as well doing this by having a semantically blocking require that is free to of course do whatever it wants as long as the execution is sequential in effect.

If there's some dynamic trickery the implementation can't figure out, just revert to blocking and nag in the console.

[brundolf]

Being imperative instead of declarative, it hinders a host of key optimizations like parallel loading and dead code elimination

[jampekka]

This is discussed in the link I posted. Bundlers manage to statically analyze CommonJS just fine. With require("stringliteral") and exports.thing = thing, that cover 99.9% or so usage, this is just as easy to statically analyze as ESM.

Saying that you need declarative for static analysis is like saying tail call optimization is impossible.

And if a stricter module system would be still required, it could have been quite easy to make compatible with, well, require.

[petre]

Require is fine but I totally hate the CJS exports. Feels like writing '90s era Perl code. On the other hand ESM dynamic imports like

  import('this').then(r => that()) 

rather suck.

[ptx]

The post dismisses native browser support (a huge benefit of ES modules) as "an utterly useless feature" that's "unavoidably slow" because of the additional roundtrips needed as the dependency tree is traversed.

But this problem has been solved by modulepreload[0], also natively supported in the browser, which lets you specify the modules upfront in the HTML, avoiding the need for additional roundtrips. Tooling could help with generating the list of preloads but is not necessary.

[0] https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes...

[jampekka]

Tooling could also analyze the dependencies and create appropriate bundles of the code to avoid roundtrips. Actually such tooling has existed for over a decade and are used by practically all but simplest Javascript applications.

I'm all for development without a compilation step. I've written hacks to resolve node modules and transpile in browser to avoid compilation. Implementing prefetching even with this hack would be less hacky than going full circle and injecting HTML tags to do the prefetching.

ESM is like XML: The problem it solves is not hard, and it doesn't solve the problem well. Actually ESM it isn't even close to solving the problem after trying for almost 15 years.

[ptx]

As you say, tooling can solve the problem. But with ES modules and modulepreload, we can now (contrary to what the post argues) also solve it without the tooling, which is an improvement.

Yes, all but the simplest applications currently use the tooling-based solutions, because there was no other way. But now that we have an alternative solution, perhaps all but the most complex applications will manage just fine without tooling, using just the built-in module support.

[jampekka]

So without tooling you need to specify the preload tags by hand. And of course import the modules in the JS.

A simple sync and/or async function/statement would had solved the problem of having modules in the browser. E.g. RequireJS solved this in 2010 or so. There were straightforward proposals for native modules even before this.

[ptx]

> So without tooling you need to specify the preload tags by hand. And of course import the modules in the JS.

Yes. This is exactly what I want. It's helpful for simple apps, and without all the toolchain complexity more apps can be simple.

ES modules solve the problems I care about:

1) Eliminating the extra roundtrips.

2) Nice and simple syntax.

3) Tooling not required.

[w3news]

ESM is a defined default you find back in ECMA specifications. That is why everybody should migratie. Node.je is also moving ESM to the default. If some systems dont do it, dont use it anymore. That it went i dont use jest anymore, and Node.js has also now a good test runner. (Useful for packages and backend systems, i think for frontend systems with eg React there are better test suits to help with special frontend stuff like the DOM)