[wonnage]

This is specifically about breaking the myth that performing expensive self-contained operations (e.g, parsing GraphQL) in a native extension (C, Rust, etc.) is always faster than the interpreted language.

The JS ecosystem has the same problem, people think rewriting everything in Rust will be a magic fix. In practice, there's always the problem highlighted in the post (transitioning is expensive, causes optimization bailouts), as well as the cost of actually getting the results back into Node-land. This is why SWC abandoned the JS API for writing plugins - constantly bouncing back and forth while traversing AST nodes was even slower than Babel (e.g https://github.com/swc-project/swc/issues/1392#issuecomment-...)

[mananaysiempre]

Interesting that both of the points you state completely contradict my experience with LuaJIT.

Parsing has always been one of the things its tracing JIT struggled with; it is still faster than the (already fairly fast) interpreter, but in this kind of branch- and allocation-heavy code it gets nowhere near the famed 1.25x to 1.5x of GCC (or so) that you can get by carefully tailoring inner-loopy code.

(But a tracing JIT like LuaJIT is a different from a BBV JIT like YJIT, even if I haven’t yet grokked the latter.)

LuaJIT’s FFI calls, on the other hand, are very very fast. They are still slower than not going through the boundary at all, naturally, but that’s about it. On the other hand, going through the Lua/C API inherited from the original, interpreted implementation—which sounds similar to what the Ruby blog post is comparing pure-Ruby code to—can be quite slow.

The SWC situation I can’t understand quickly, but apart from the WASM overhead it sounds to me like they have a syntax tree that the JS plugin side really wants to be GCed in the GC’s memory but the Rust-on-WASM host side really wants to be refcounted in WASM memory, and that is indeed not a good situation to be in. It took a decade or more for DOM manipulation in JS to not suck, and there the native-code side was operating with deep (and unsafe) hooks into the VM and GC infrastructure as opposed to the WASM straitjacket. Hopefully it’ll become easier when the WASM GC proposal finally materializes and people figure out how to make Rust target it.

In any case, it annoys me how hard it is in just about any low-level language to cheaply integrate with a GC. Getting a stack map out of a compiler in order to know where the references to GC-land are and when they are alive is like pulling teeth. I don’t think it should be that way.

[aardvark179]

There is a very big difference between a simple FFI system and the sort of C interface offered by Ruby and Node. Those interfaces allow objects to be passed to the native code, and the native can then do pretty much anything to the language run state. This is great if you want a C library that can do anything your higher level language could do, but it also means the JIT has to treat all those calls as impenetrable barriers that cannot be optimised through, so even a small C call can prevent the rest of your application from being optimised.

We got round this in TruffleRuby by running C extensions through an LLVM Bitcode interpreter that was part of the same framework as the Ruby interpreter and allowed them to be JITted together, but that had other downsides, and wasn’t great for things like parsers which had huge switch statements.

[mike_hearn]

Yes but in this case the TruffleRuby approach would fix the Shopify issue, I think? And if by downside you mean longer warmup times that's an issue for YJIT or any other JIT too, so how much of a downside it is depends a lot on the nature of the deployment.

[byroot]

Shopify bigger repos are deployed pretty much every 30 minutes. As you point out most JITs struggle in these conditions.

But YJIT warms up extremely fast, and is able to provide real world speedup to these services almost immediately.

[vidarh]

I think a parser is perhaps the example of where resorting to a compiled extension can be beaten by something more JIT-favourable.

In a language like Ruby it tends to be heavily dominated by scanning text, and creation of objects, and 1) you can often speed it up drastically by reducing object creation. E.g. here is Aaron writing about speeding up the GraphQL parser partly by doing that[1], 2) creating Ruby objects and building up complex structures in the C extension is going to be almost exactly as slow in the C extension as in the Ruby, 3) the scanning of the text mostly hits the regexp engine which is already written in C.

(That said, I heavily favour not resorting to C-extensions unless you really have to; even without going as far as some of Aaron's more esoteric tricks for that parser you can often get a whole lot closer that you think, and the portion you need to rewrite if you still have to might well turn out to be much smaller than you'd expect)

[1] https://tenderlovemaking.com/2023/09/02/fast-tokenizers-with...

[danmur]

I think 'magic fix' could be replaced with 'fun thing to do'

[yebyen]

I spent a while rewriting a tiny bit of some useful Ruby in Rust and integrating it via Wasmer, and I definitely think my time expenditure is better classified as "fun thing to do" than "magic fix".

https://ossna2023.sched.com/event/1K55z/exotic-runtime-targe...

https://www.youtube.com/watch?v=EsAuJmHYWgI

My goal was to determine: how will I use Wasm as a Rubyist? Spoiler: I did not intend to use Rust, but embedding Ruby in a Wasm and running it from within Ruby proved to be a fool's exercise. I have a feeling that some of the claims I made in this talk about there being "no theoretical benefit" to running Ruby in Wasm in Ruby will quickly be proven incorrect. But I'm not expecting it to be faster.

If it's ever faster, that will be filed under "surprising results"

Michael Yuan who obviously knows a lot more about Wasm than I do addressed this in his talk as well, but not from the perspective of a JIT necessarily, although I guess you could consider local machine compile target optimization on the target machine at runtime as a type of "just in time optimization" it's not really (it's just the regular ahead-of-time type of optimization, but not fumbled and the majority of benefits immediately getting lost in such a way as it usually is...)

https://www.youtube.com/watch?v=kOvoBEg4-N4

The spoiler from that talk (for me anyway) was finding out that you do see surprising results sometimes, and sometimes it's due to a pathological case that (a) happens all the time, and (b) does not have a readily obvious solution in the form the problem regularly takes. Like linux distros shipping generic binaries "target-optimized" for the lowest common denominator on any given particular architecture dist target, because the binaries they ship obviously have to run anywhere.

(Recap of the conversation we had off-stage after the Q&A ended: We don't know when we say "x86-64" what modern opcode targets that means we really have available, so we have to ship a binary with only the oldest opcodes that are guaranteed to be available on any similar chip that we intend to support. Web Assembly uses a compiler on the target machine, cranelift, to translate from a platform independent binary to a platform-specific one. Running the compiler takes a while, but we can do it ahead of time. Will we come out ahead in the end?)

All of this stuff is certainly very fun to reason about :D