[constexpr]

I'm the author of esbuild. I think this is a very smart, practical choice and is one of the approaches that has the best chance of succeeding (the other being a semi-automated porting tool instead of a purely manual port). I'm very excited to see someone take this approach. This is how I would do it if I were to do it. Web developers around the world are hoping this works out :)

The hard part about this project isn't doing the port but keeping up with the constant changes from the TypeScript team, especially if the TypeScript team isn't invested in helping you out. Staying as close as possible to the source language like this will be a big win there.

One of the big benefits of a native language like Go aside from lack of a JIT is the ability to easily add shared-memory parallelism. I haven't studied the TypeScript compiler internals so I'm not sure how easy it is to parallelize their type checker. However, I assume that it would be relatively straightforward to parallelize the parsing of JavaScript files, so I'd hope for at least be a big speedup there.

[msoad]

Unfortunately TypeScript is doing all of type-checking synchronously mostly due how it needs to build a global symbols list first. There are some hints that they might move towards more parallel processing but it requires a lot of major changes

https://github.com/microsoft/TypeScript/issues/30235

[constexpr]

Yeah I was afraid of that. Ideally it would be possible to do type checking in parallel at the function level or something, but TypeScript allows you to omit the return type and then infers the return type from the function body which adds additional dependencies and makes that more complicated to parallelize. I wonder if it would still be possible though. Maybe just for functions where the return type and all argument types are present? Then the approach would be sort of outside-in where you type check the unparallelizable global stuff serially first and then type check the remaining isolated function body stuff in parallel afterward.

[zja]

Love ESBuild! Convert a relatively large project from webpack to it today, really easy to do & much faster build times now!

[vips7L]

> the ability to easily add shared-memory parallelism

I’m honestly not sure if there is any popular language where this isn’t true?

[constexpr]

The problem is that you can't use shared-memory parallelism in TypeScript. If you could, then the most straightforward way of speeding up the TypeScript compiler would be to parallelize the compiler, not to port it.

[vips7L]

I just don’t see how Go is more uniquely suited to do this than any other language with the same feature and admittedly I may have read too much into your comment.

[constexpr]

That comment wasn't about Go exclusively. My point was that porting single-threaded code to a native language doesn't necessarily cause huge order-of-magnitude speed gains, especially if you plan on sticking close to the original structure of the code to make porting future changes easy, but that in this specific case a port sounds likely to realize significant gains (which is very exciting!).

Shared-memory parallelism is necessary but not sufficient. Ideally you'd be able to only make some small changes in a few spots to greatly increase parallelism. I'm hopeful that TypeScript's AST parsing is a good candidate since ASTs can typically be constructed independently from the rest of the program state. Lowering and printing ASTs is also likely parallelizable. It would also be great if it was also possible to parallelize parts of the type checker somehow, but that sounds harder. I had some ideas about that in another comment. Anyway I think this means that it's likely that a port of the TypeScript compiler could realistically lead to significant speedups and may even lead to extreme speedups.

As far as language choice, it sounds like due to maintenance reasons the decision space is narrowed to native, garbage collected languages similar in object model to TypeScript but with parallelism primitives. Some candidates could be at least Go, JVM-based languages, or .NET-based languages. I think those can all be ahead-of-time compiled to a static binary? Another consideration is how easy/compact/fast/cross-platform binaries are. Go makes this trivial but I haven't used JVM or .NET AOT builds myself so I don't know how they compare in e.g. executable size or startup overhead. One advantage of picking Go is that esbuild has already demonstrated that Go is an effective choice for this problem domain, which may mean less work/research/unknowns. A disadvantage of Go here might be that the object model is different than TypeScript vs. something like Java or C#. But all else being equal the choice comes down to personal preference.

[jpochtar]

node.js Workers + SharedArrayBuffers?

I'm not sure how well supported they are / haven't used them, but I think they enable this?

[constexpr]

Yes that allows for shared memory. But the TypeScript compiler is written in TypeScript, and you can't put JavaScript objects (such as the TypeScript AST) in a SharedArrayBuffer. So you'd have to port the TypeScript compiler to another language that can target multithreaded WASM to do this.

You could also try sending JS objects between workers instead of using SharedArrayBuffer. But that copies the memory, which can take a significant amount of time, and of course also excludes certain kinds of advanced parallelism that require shared memory (instead of copied memory). Another drawback of using the JS worker model like this is that having multiple JS VMs probably means fewer CPU cores available due to there being multiple independent garbage collectors. I observed an effect like this when I tried to parallelize esbuild's JavaScript plugins. Running them in separate workers seemed to cause the gains from parallelism to stop when there were half as many workers as CPUs. I'm guessing is because half of the CPUs are busy collecting garbage for the other half. Using a language like Go with real shared memory presumably makes it easier for fewer resources to be spent on garbage collection.

[msoad]

Folks at Parcel wrote serializers that uses SharedArrayBuffer for their core stuff like graph[1]. I agree with parent comment that this didn't have to happen in Go, JS would've been fine if they were willing to depart from the source enough.

[1] https://github.com/parcel-bundler/parcel/tree/v2/packages/co...

[sealeck]

Python? JavaScript? Ruby?

[joshxyz]

love esbuild, ditched webpack and rollpack since then. been a fan of golang too since then.