[gavinray]

I had a neat conversation with @fniephaus on the GraalVM slack, where I was curious about how the performance of the Native Image mode could be almost an order of magnitude better in the Game of Life demo than JVM JIT mode.

He clarified that the GIF showed only the first N seconds of the program running, where the AOT binary required no warmup. But what was really interesting, is his comment about how AOT mode is still able to perform potentially slightly better than JIT:

  > "The GIF is showing the first n seconds, and the JIT just needs noticeable more time to warm up. But even at peak, AOT can outperform JIT although not by an order of magnitude of course."

I asked how this was possible and he shared a great tweet by @AlinaYurenko:

https://twitter.com/alina_yurenko/status/1582772754902052864

  > AOT can be faster than JIT, because:
  > - in AOT 100% of the code is compiled (on JIT cold code can still be interpreted)
  > - some optimizations are only possible under a closed-world assumption (AOT)
  > - AOT can dedicate time and resources to perform more expensive optimizations

[kaba0]

I don’t really agree with these points regarding performance speed (of course AOT is a cool technology that has its uses):

> - in AOT 100% of the code is compiled

If the code hasn’t been run enough times to become eligible for JIT compilation, it likely doesn’t contribute any significant time to the whole runtime, so I doubt it would be a meaningful change.

> some optimizations are only possible under a closed-world assumption (AOT)

Which a JIT compiler is more than free to assume much more strictly than an AOT one can? Like, if it sees that only a single implementing class of an interface is loaded it can assume that every virtual method call can be replaced by a static one. Upon a class load, this assumption can be revisited and the native code can be deoptimized in cases. In an AOT compiler you have to optimize based on the worst case, while JIT compiler may avoid loading that class based on some dynamic property. Also, Graal is not only an AOT compiler, it is also a JIT compiler with.. closed-world assumptions, so it is quite meaningless comparison.

The last point is true on paper, but as far as I know it is not true that JIT compilers produce worse code, and even if it is it is not due to lack of time/resources.

But I just wanted to refute the performance claims — Graal Native executables do start up much faster and have significantly less memory usage, which are worthwhile goals in themselves. But most Java code will perform better under a JIT compiler (which can be Graal’s as well)

[gavinray]

  > " But most Java code will perform better under a JIT compiler (which can be Graal’s as well)"

This was my understanding, for any comparison of world-class JIT compilers vs AOT codegen. But, fniephaus knows his stuff and there are some compelling particular examples/benchmarks given. I haven't taken the time to do an exhaustive comparison.

It does seem sort of counterintuitive though, doesn't it? Like, what's the point of an optimizing JIT then?

[kaba0]

I’m sure there are examples where Graal Native will beat the JIT version of the same program (e.g. it often does more throughout escape analysis, also, smaller object allow more data to fit inside cache, etc), but in case of a “typical” application I would wager that the JIT approach is better. Maybe that’s just the way idiomatic JVM code is written?

[pjmlp]

The problem with deoptimization is the performance hit every time it takes place, which leads to a saw graph similar to when there is too much GC going on.

[kaba0]

Sure, but class loading only changes assumptions in one direction for example — in the aforementioned example it would cause only one deoptimization after which the code won’t be worse off than the AOT compiled one. So the JIT case before class load may perform better, and afterwards will perform the same.

[pjmlp]

Classes can also be unloaded.

What JIT does definitely better is when coupled with PGO data that survives between execution sessions.