[pcwalton]

> Many places in my code had switch statements that branched over all AST nodes and in Rust that compiles to code which uses stack space proportional to the total stack space used by all branches instead of just the maximum stack space used by any one branch: https://github.com/rust-lang/rust/issues/34283.

Can you work around this by using separate functions for the branches? This will have other benefits for compile time as well. Generally, small functions are better for compile time, because some compiler passes are not O(n), including very basic ones like register allocation.

For large switch statements I do this for readability reasons, because I try to keep my functions small.

> Based on a profile, it looked like the Go version was faster because GC happened on another thread while Rust had to run destructors on the same thread.

Have you tried using jemalloc? It can help a lot for situations like this.

[constexpr]

> Have you tried using jemalloc? It can help a lot for situations like this.

Huh, I thought Rust already used jemalloc by default. I looked it up and it looks like it was removed relatively recently. When I was doing this experiment, I was using a version of Rust that included jemalloc by default so that 10% number already uses jemalloc. I remember this because I also thought of trying to speed up the allocator.

Like I said above, I profiled both Go and Rust and the 10% slowdown with Rust appeared to be running destructors for the AST. I think the appropriate solution to this would be some form of arena allocator instead of changing the system allocator. But that gets even more complicated with lifetimes and stuff.

> Can you work around this by using separate functions for the branches?

Yeah, I could have tried restructuring my code to try to avoid compiler issues. But this would have been even more time spent working around issues with Rust. Go was better than Rust by pretty much every metric that mattered for me, so I went with Go instead.

It's too bad because I was initially super excited about the promise of Rust. Being able to avoid the overhead of GC while keeping memory safety and performance is really appealing. But Rust turned out to be not a productive enough language for me.

[pcwalton]

> I think the appropriate solution to this would be some form of arena allocator instead of changing the system allocator. But that gets even more complicated with lifetimes and stuff.

If you're doing arena allocation in a compiler you might as well just leak all your allocations (which you could get with bumpalo with a 'static lifetime); then you won't have to deal with lifetimes at all.

> Yeah, I could have tried restructuring my code to try to avoid compiler issues.

Well, my point is that it would be good for readability to restructure your code in that way even in Go. 500-line functions are hard to read.

> But this would have been even more time spent working around issues with Rust. Go was better than Rust by pretty much every metric that mattered for me, so I went with Go instead.

I find the opposite to be true, especially for compilers. It's hard for me to go back to a language without pattern matching and enums (much less generics, iterators, a package ecosystem, etc.). The gain of productivity from GC and compile times is not worth Go's loss in productivity in other areas for me. But reasonable people can disagree here.

[constexpr]

> If you're doing arena allocation in a compiler you might as well just leak all your allocations (which you could get with bumpalo with a 'static lifetime); then you won't have to deal with lifetimes at all.

I considered this but it's a very limiting hack. Ideally esbuild could be run in watch mode to do incremental builds where only the changed files are rebuilt and most of the previous compilation is reused. Memory leaks aren't an acceptable workaround to memory allocation issues in that case. While I don't have a watch mode yet, all of esbuild was architected with incremental builds in mind and the fact that Go has a GC makes this very easy.

> 500-line functions are hard to read.

I totally recognize that this is completely subjective, but I've written a lot of compiler code and I actually find that co-locating related branches together is easier for me to work with than separating the contents of a branch far away from the branch itself, at least in the AST pattern-matching context.

> It's hard for me to go back to a language without pattern matching and enums

I also really like these features of Rust, and miss them when I'm using other languages without them. However, if you look at the way I've used Go in esbuild, interfaces and switch statements give you a way to implement enums and pattern matching that has been surprisingly ergonomic for me.

The biggest thing I miss in Go from Rust is actually the lack of immutability in the type system. To support incremental builds, each build must not mutate the data structures that live across builds. There's currently no way to have the Go compiler enforce this. I just have to be careful. In my case I think it's not enough of a problem to offset the other benefits of Go, but it's definitely not a trade-off everyone would be comfortable making.

[pcwalton]

> interfaces and switch statements give you a way to implement enums and pattern matching that has been surprisingly ergonomic for me.

With no exhaustiveness checking (also no destructuring, etc.)

I should also note that you can change the default stack size in Rust to avoid overflows, though there should be a bug filed to get LLVM upstream stack coloring working. It's also possibly worth rerunning the benchmark again, as Rust has upgraded to newer versions of LLVM in the meantime.

[no_wizard]

This really reeks to me of trying to shoehorn Rust into the solution rather than it being an organic fit to the problem space.

I don’t see the value in this level of thinking here. Why should any developer go through this much hassle when they have a perfectly good solution that, really, I’m not seeing any discussion about this that actually highlights issues in the approach to using ago for this sort of thing

[pcwalton]

Generally, Rust "should" be faster, because it spends a lot of time on optimizations that Go doesn't do. That's what you're paying for in compile times. If Go is faster on some CPU bound workload despite doing a lot less optimization, that's interesting. (I should note that this is not the norm.)

[esimov]

I remember an article from Figma or even possible from you, where you mentioned that you have rewritten in Rust a tool originally written in node.js. And I remember, back at that time (I think it was 2 years ago) you were very exited about the language. How does it compare to the current situation? What made you to think that Go is a better and enjoyable language than Rust? Is it the faster GC?

A recently published article from a guy working at Discord throw a real flameware, because he was arguing about the opposite: Rust is a better and faster language then Go, but he based his assumption on very old version of Go (1.9) where the GC was way slower then in the current versions.

[no_wizard]

Why work around it when the Go version worked as is?

I’m not sure I really see the benefit of working around it when Go fit the use case as is

[pcwalton]

Well, I'd like to know if there are things we can do to improve things in Rust.

[no_wizard]

Yes, fix this open issue

https://github.com/rust-lang/rust/issues/34283

That would help immensely.

[pcwalton]

Do we know that Go uses stack coloring, actually? It's a relatively uncommon optimization.