[mike_hearn]

In theory once Graal adds support for it, any Graal/Truffle-compatible language can benefit.

IMHO it's only JVM+Graal that can bring this to other languages. Loom relies very heavily on some fairly unique aspects of the Java ecosystem (Go has these things too though). One is that lots of important bits of code are implemented in pure Java, like the IO and SSL stacks. Most languages rely heavily on FFI to C libraries. That's especially true of dynamic scripting languages but is also true of things like Rust. The Java world has more of a culture of writing their own implementations of things.

For the Loom approach to work you need:

a. Very tight and difficult integration between the compiler, threading subsystem and garbage collector.

b. The compiler/runtime to control all code being used. The moment you cross the FFI into code generated by another compiler (i.e. a native library) you have to pin the thread and the scalability degrades or is lost completely.

But! Graal has a trick up its sleeve. It can JIT compile lots of languages, and those languages can call into each other without a classical FFI. Instead the compiler sees both call site and destination site, and can inline them together to optimize as one. Moreover those languages include binary languages like LLVM bitcode and WASM. In turn that means that e.g. Python calling into a C extension can still work, because the C extension will be compiled to LLVM bitcode and then the JVM will take over from there. So there's one compiler for the entire process, even when mixing code from multiple languages. That's what Loom needs.

At least in theory. Perhaps pron will contradict me here because I have a feeling Loom also needs the invariant that there are no pointers into the stack. True for most languages but not once C gets involved. I don't know to what extent you could "fix" C programs at the compiler level to respect that invariant, even if you have LLVM bitcode. But at least the one-compiler aspect is not getting in the way.

[kaba0]

With Truffle you have to map your language’s semantics to java ones. I am unfortunately out of my depth on the details, but my guess would be that LLVM operates here with this in mind in a completely safe way (I guess pointers to the stack are not safe) so presumably it should work for these as well.

[mike_hearn]

Not exactly, no. That's the whole point of Truffle and why it's such a big leap forward. You do not map your language's semantics to Java semantics. You can implement them on top of the JVM but bypassing Java bytecode. Your language doesn't even have to be garbage collected, and LLVM bitcode isn't (unless you use the enterprise version which adds support for automatically converting C/C++ to memory safe GCd code!).

So - C code running on the JVM via Sulong keeps C/C++ semantics. That probably means you can build pointers into the stack, and then I don't know what Loom would do. Right now they aren't integrated so I guess that's a research question.

[kaba0]

Perhaps I wasn’t clear. I do know that Truffle works by writing an AST interpreter for another language, but to achieve the best performance you have to map/reuse existing java constructs. E.g. I have read that perhaps Ruby uses java exceptions in a not too idiomatic way, but this is what Graal can later optimize to very good code.

My way out of depth idea with Sulong is that it uses small heap-allocated regions for every manual memory usage (it even has a Managed mode in Enterprise).

[mike_hearn]

You use Java constructs to implement the interpreter, but that doesn't mean the language itself has to be mapped to Java constructs, any more than writing an interpreter in C means your language has to be mapped to C semantics.

Sulong uses a standard C-style heap in the open source version. In EE they (can) trap malloc/free and re-point it towards the GCd heap. They also do bounds checking on pointer de-references. It's actually amazingly cool but unfortunately, EE is expensive enough in dollar terms that it gets ignored. I don't know of anything that uses it for real.