[workingjubilee]

Your experience is not perfectly transferable. JITs have it easy on this because they've already gathered a wealth of information about the actually-executing-on CPU by the time they generate a single line of assembly. Calls appear on the hot path more often in purely statically compiled code because things like the runtime architectural feature set are not known, so you often reach inlining barriers precisely in the code that you would most like to optimize.

[pizlonator]

LLVM inlines even more than my JIT does.

The JIT has both more and less information.

It has more information about the program globally. There is no linking or “extern” boundary.

But whereas the AOT compiler can often prove that it knows about all of the calls to a function that could ever happen, the JIT only knows about those that happened in the past. This makes it hard (and sometimes impossible) for the JIT to do the call graph analysis style of inlining that llvm does.

One great example of something I wish my jit had but might never be able to practically do, but that llvm eats for breakfast: “if A calls B in one place and nothing else calls B, then inline B no matter how large it is”.

(I also work on ahead of time compilers, though my impact there hasn’t been so big that I brag about it as much.)

[kevincox]

> the JIT only knows about those that happened in the past.

This is typically handled by assuming that all future calls will be representative of past calls. Then you add a (cheap) check for that assumption and fall back to interpreter or an earlier JIT that didn't make that assumption.

This can actually be better than AOT because you may have some incredibly rare error path that creates a second call to the function. But you are better off compiling that function assuming that the error never occurs. In the unlikely case it does occur you can fall back to the slower path and end up faster overall. Unless the AOT compiler wants to emit two specializations of the function the generated code needs to handle all possible cases, no matter how unlikely.

Of course in practice AOT wins. But there are many interesting edge cases where a JIT can pull off an optimization that an AOT compiler can't do.

[saagarjha]

The people who write JITs also write a bunch of C++ that gets statically compiled.