[chombier]

This article is about a technique "Copy-and-Patch" for just-in-time (JIT) compilation that is both fast to compile, produces reasonably efficient machine code, and is easier to maintain than writing assembly by hand.

The section "Copy-and-Patch: the Art of Repurposing Existing Tools" describes the heart of the method, which is to use an existing compiler to compile a chunk of c/c++ code corresponding to some bytecode instruction, then patch the resulting object file in order to tweak the result (e.g. to specify the instruction operands) in a similar fashion to symbol relocation happening during link.

Given a stream of bytecode instruction, the JIT compilation reduces to copying code objects (named "stencils") corresponding to bytecode instructions from a library of precompiled stencils then patching the stencils as needed, which is very fast compared to running a full-blown compiler like LLVM from the syntax tree.

Of course, the resulting code is slower than full-blown Ahead-of-Time (AOT) compilation, but the authors describe a few tricks to keep the execution speed within a reasonable margin of AOT. For instance, they leverage tail calls to replace function calls with jumps, compile sequences of frequently associated bytecode instructions together, and so on.