[lemming]

Great article, as always.

There is one thing that I think is important to bear in mind when discussing inlining, especially in the context of Clojure. This is that once a function has been inlined, you can no longer update the definition of that function in the REPL and have that update the behaviour of functions which use it, unless you recompile those as well. This is not a criticism of course, it’s just part of the natural tension between dynamism and performance.

[eigenspace]

Julia actually has some really cool machinery for handling this that I would encourage other JIT languages to copy.

Whenever you call a function, that function and any calls in that call stack occur in a 'fixed world age'. Within a given world-age, method tables and global constants are all fixed, and the langauge can be analyzed like it's statically typed (there are escape hatches like `invoke_in_world`, and `invokelatest`)

Between world-ages, things are allowed to change. When a function calls another function, we add a 'backedge' from the caller to the callee.

So if I have `f(x) = g(h(x))`, and I redefine `h`, we then say it's no longer valid, and then we look at the backedge that leads from `h` to `g` and say the old definition of `g` is also no longer valid, and then we go from `g` to `f` and also invalidate the old definition of `f`.

This means that once `f` is called in a new world age (the world-age gets incremented every time a new method is (re)defined, or if a global const is changed / defined), the compiler knows that it has to recompile `f`, `g`, and `h`. What's especially cool is that this system works regardless of inlining, and it allows us to safely do all sorts of interproceedural optimizations, but in a JIT compiled language.

[lemming]

That is very cool indeed. Are there limitations that this imposes? Is Julia a whole world compiler or does it support partial compilation?

[eigenspace]

There's two main limitations:

1. If you try and re-define a global constant or add new methods inside a running program using `eval` or whatever, then your running program won't see those changes until it advances the world-age (i.e. either by using `invokelatest`, or by returning to the top-level scope). Note though that things like closures and defining functions within functions is fine, you just can't do an arbitrary `eval` to define something completely dynamially

2. Method invalidations can cause a lot of compilation latency. If you load a package that invalidates a bunch of already compiled methods, then those methods will later need to be recompiled, which means you hit some more compiler latency than expected. These invalidations can have false postives too, so sometimes more methods get invalidated than you'd want

__________________________

> Is Julia a whole world compiler or does it support partial compilation?

On the LLVM side, we only do partial compilation. Every function method specialization in each different world (modulo inlining) is its own LLVM module that gets compiled in parallel by LLVM. Non-inlined function calls then involve linking these modules.

On the julia-side with our own custom internal IRs though, that's where we perform whole-world style interproceedural optimizations and inlining before handing the individual compilation units to LLVM. At least if I'm using "whole world" right here. What I mean is essentially everything statically known to be reachable from a compilation unit's entry-point given its signature. If by "whole world" you mean compiling every possible method signature, that's not possible in julia at all, because the space of possible method specializations is infinite due to parametric types.

We generally get the best of both worlds with these two approaches (at the cost of just using a lot of space to store all the different possible specializations and all of our differnt IRs and different pieces of machinery).

[lemming]

That is really interesting, and very clever - thanks for the details! What you describe is indeed what I meant by whole world, I'm not sure if there's a better term of art for it.

[Jeaye]

Hey lemming! You're right, which is why it should be used sparingly. Since clojure.core is compiled (on the JVM) with direct linking, reacting to var changes isn't an intended concern, since they're not going to work properly throughout any clojure.core code using that var. This makes it a good candidate ns for inlining things. But users shouldn't just be doing this for their normal application vars without giving it due consideration.

[thfuran]

Does that not happen automatically? I know there are contexts in which jvm will deoptimize inlining and recompile, like in response to class loading that causes a call site that was previously provably monomorphic to no longer be.

[lemming]

No, it doesn't. In JVM Clojure's case, the vars are usually compiled to the moral equivalent of a global variable holding a pointer to a function. This allows you to update the function if the developer redefines it in the REPL, but it comes at a performance cost (the JVM can't inline it or otherwise optimise it). Clojure also allows you to compile with "direct linking", e.g. for production deployments, where you know you're unlikely to be wanting to dynamically update the code. In those cases defns are compiled down to static methods which call each other - much faster since the JVM can perform its magic with them, but you can't update them at the REPL.

I'm unsure exactly how jank works WRT this tradeoff, but the article makes it sound like it's closer to the direct linking version, but with the inlining etc being done by jank rather than the JVM. I don't know if this is only for AOT or also in JIT cases.

[arvyy]

> the vars are usually compiled to the moral equivalent of a global variable holding a pointer to a function. This allows you to update the function if the developer redefines it in the REPL, but it comes at a performance cost (the JVM can't inline it or otherwise optimise it)

might be out of my depth but I find it surprising; I thought compilation through invokedynamic should be able to handle redefinition while still allowing inlining and other jit optimizations

[lemming]

Clojure (AFAIK) does not use invokedynamic, except perhaps in the latest version for some of the new interop stuff. It still officially supports JVM 1.8 bytecode. It’s a language which greatly values stability and backwards compatibility, so it’s been very slow to adopt newer JVM features.

[kbolino]

Though there may be other reasons not to use it, invokedynamic is not a new feature of the JVM. If they're targeting 1.8 binary compatibility, they certainly have it at their disposal, since it landed in 1.7.

[lemming]

I can't speak for the core team, but from memory invokedynamic took a long time to become performant. So if you still want to support older JVM versions, the performance will be pretty terrible on those older systems if invokedynamic is used for something as integral as var lookups.

[mccoyb]

Is that really true? Can't you track invalidations via a dependency graph?

Right, as you said, you'd have to recompile dependents.

[adgjlsfhk1]

That's what Julia does. It works pretty well.