[foldr]

>Doing _anything_ to reduce this improves performance dramatically

It does if you ignore the overhead of JIT compilation itself. However, my understanding is that writing a JIT implementation that performs better than a good interpreter is surprisingly difficult. You have to have a lot of complicated logic for tracking hotspots and using JIT judiciously in short-running scripts.

[jashmatthews]

Hmmm, not really. I think that's somewhat true for JS and webpages, but they're not the same as server-side apps.

It's the instruction dispatch overhead that's the real unavoidable problem. LuaJIT, for example, uses a bunch of tricks to minimize it in the bytecode VM, and it's significantly faster than the standard Lua VM but still far, far slower than basic JIT compilation.

[foldr]

Right, but historically there are lots of instances of projects that abandoned JITs because they didn't get a performance improvement. JIT compilation reduces instruction dispatch overhead, but it also, unless accompanied by sophisticated profiling techniques, adds the overhead of JIT compilation time, which can easily swamp the improvements.

Lua JIT is one of the most sophisticated dynamic language JITs out there, so it's hardly evidence that a simple implementation of a JIT will perform better than a good bytecode interpreter.

The problem is less acute for server side apps because the programs run for a long time, so that the initial compilation overhead is insignificant. However, there's a reason that you need a JIT to make Ruby fast rather than an ahead of time compiler. Ruby has so few compile-time guarantees that you need to do a lot of dynamic specialization to get really significant performance improvements. So compilation might still be triggered even after a script has been running for a long time.

I'd add that PyPy, which is also very sophisticated, is often not much faster than CPython, and in fact is slower for some types of code. Writing good JIT-based implementations for dynamic languages is really a tough problem. See e.g. the following post for some explanation of why:

http://faster-cpython.readthedocs.io/notes_2017.html

[jashmatthews]

> Right, but historically there are lots of instances of projects that abandoned JITs because they didn't get a performance improvement. JIT compilation reduces instruction dispatch overhead, but it also, unless accompanied by sophisticated profiling techniques, adds the overhead of JIT compilation time, which can easily swamp the improvements.

Yes.

> Lua JIT is one of the most sophisticated dynamic language JITs out there, so it's hardly evidence that a simple implementation of a JIT will perform better than a good bytecode interpreter.

I meant that even a basic JIT can offer the same speedup as LuaJIT's interpreter, and a lot more work went into the latter.

> The problem is less acute for server side apps because the programs run for a long time, so that the initial compilation overhead is insignificant. However, there's a reason that you need a JIT to make Ruby fast rather than an ahead of time compiler. Ruby has so few compile-time guarantees that you need to do a lot of dynamic specialization to get really significant performance improvements. So compilation might still be triggered even after a script has been running for a long time.

The initial results of MJIT for simply removing the instruction dispatch overhead and doing some basic optimizations are a 30-230% performance increase on a small but real-world benchmark. No type specialization and specular optimization required.

> I'd add that PyPy, which is also very sophisticated, is often not much faster than CPython, and in fact is slower for some types of code. Writing good JIT-based implementations for dynamic languages is really a tough problem. See e.g. the following post for some explanation of why:

Most of the discussion about PyPy is completely irrelevant for the discussion about MJIT. PyPy isn't a method JIT. PyPy traces the interpreter itself and tries to produce a specialized interpreter. It works even worse at optimizing Ruby code via Topaz.

[chrisseaton]

> It works even worse at optimizing Ruby code via Topaz.

Topaz was easily the fastest Ruby JIT before TruffleRuby, beating the JRuby and Rubinius JITs. It was very impressive.

[jashmatthews]

True! I just checked again and Topaz is indeed almost twice as fast as CRuby on optcarrot. I think I got it mixed up with the non-JIT Rubinius numbers.

It'a shame Topaz was never really "finished".

[chrisseaton]

One of the main Topaz developers now works on the Python equivalent of TruffleRuby.

[foldr]

>The initial results of MJIT for simply removing the instruction dispatch overhead and doing some basic optimizations are a 30-230% performance increase on a small but real-world benchmark. No type specialization and specular optimization required

So, this amounts to a small improvement for some types of code. Indeed, it is "easy" to get that by "just" using some basic JIT techniques. The trick is to get consistently better performance across the board. Relevant tweet at https://medium.com/@k0kubun/the-method-jit-compiler-for-ruby...:

>I've just committed the initial JIT compiler for Ruby. It's not still so fast yet (especially it's performing badly with Rails for now), but we have much time to improve it until Ruby 2.6 (or 3.0) release.

[jashmatthews]

> The trick is to get consistently better performance across the board.

This will come with the rest of the opimizations Takashi has planned for Ruby 2.6. Ruby-Ruby method inlining, which is almost finished, is a huge one for improving Rails performance. IMHO there's no real point talking about Rails until it's working in some form.

> >I've just committed the initial JIT compiler for Ruby. It's not still so fast yet (especially it's performing badly with Rails for now), but we have much time to improve it until Ruby 2.6 (or 3.0) release.

It turned out this wasn't even testing MJIT with Rails because https://twitter.com/samsaffron/status/963219086833434624

[foldr]

I'm not saying that MJIT (or whatever implementation) will never be fast. I'm just saying that in general, it is not trivial to get performance improvements by writing a JIT. My original comment said nothing about Ruby.