Ten years ago was two whole technological generations ago in the implementation of OpenJDK's GCs. OpenJDK now has a maximum pause time of under 1ms for heaps up to 16TB.
I really strongly doubt that GC is a bottleneck for Erlang programs on either the BEAM or the JVM - the sophistication of the scheduler, and the way various language primitives interact with it, is where the BEAM is almost certainly gaining an edge over the JVM. That said, I'm sure there are a subset of programs that _would_ be faster on the JVM, just depends on what metrics are being compared.
> the sophistication of the scheduler, and the way various language primitives interact with it
That was brought over to the JDK six months ago. The JDK can now spawn millions of Erlang-like processes ("virtual threads") per second.
Erlang is a great inspiration and it does incredibly well with the development resources available to it, but it's hard to compete with the level of engineering investment in the JDK and its state-of-the-art GCs, optimising JIT compilers, and low-overhead in-production tracing and profiling.
The BEAM is so much more than just a green thread runtime - and it is not straightforward at all to just "bring it over" to another virtual machine, the entire BEAM VM is designed around the scheduler, and core features such as signals (used most notably for links/monitors, but also for a number of other system features) and messaging, are deeply integrated with the scheduler; as are system tasks, execution of NIFs (natively-implemented functions), and more. Furthermore, the schedulers are adaptive to the amount of work in the system, and the behavior of the other schedulers, i.e. they aren't just simple work-stealing queues for green threads. Over 20 years of engineering effort have been invested in this system, and it is highly optimized for the types of workloads that Erlang is used for.
In my opinion, trying to replicate ERTS on top of another virtual machine either requires making that virtual machine more like the BEAM, or will end up always playing catch up with the BEAM itself in some aspect. That's just my two cents though.
I am well familiar with BEAM, and my fascination with it ~20 years ago was one of the things that inspired me to become a VM engineer. Bringing that over to the JDK was, indeed, a large effort, but we are a large, well-funded team.
And, as a JDK engineer with Erlang experience, I'm just sayin' that as of the past six months, the JDK has all the primitive building blocks required for a very efficient Erlang runtime. Even the thread scheduler is not built into the JVM but written in Java, and will soon be made completely pluggable. Any scheduling algorithm that Erlang wishes to employ can be provided as a library on top of the JVM. If I've missed anything, we'll consider adding it.
It's been a dream of mine to port BEAM to the JDK to give Erlang much better performance and greater visibility. Unfortunately, this work can't be funded as there's insufficient demand for Erlang. If the Erlang community does it, however, it could increase its evolution velocity, improve performance, and give it higher visibility.
BEAM doesn't support mutation, the JVM does. that's a big plus in the BEAMs corner.
Also when two BEAM VM speak to each other its as if they are just one machine. no serialization de serialization of data, no protobufs, etc. It's all just primitives being sent between processes regardless of their location.
> BEAM doesn't support mutation, the JVM does. that's a big plus in the BEAMs corner.
The JVM supports languages with mutation or without. That x86 supports mutation doesn't hinder, say, Haskell from being efficiently compiled to it.
> Also when two BEAM VM speak to each other its as if they are just one machine. no serialization de serialization of data, no protobufs, etc. It's all just primitives being sent between processes regardless of their location.
There is no need for that protocol to be built into the VM . The JVM is a lower-level VM than BEAM (parts of the runtime, like the thread scheduler and even one of the JITs are written in Java).
I have the utmost respect for Erlang's design, and have borrowed inspiration from it. It is truly a towering achievement. But BEAM simply doesn't have anywhere near the level of investment required to be a leading platform. I wish there was more demand for it, but there isn't.
It's important to highlight some key differences between the JVM and the Erlang VM (BEAM) when it comes to accommodating language semantics, fault tolerance, and process isolation.
The JVM is designed as a general-purpose VM, aiming to accommodate a wide variety of languages with different semantics. This broad scope can sometimes lead to compromises when it comes to optimizing for specific language features or use cases. On the other hand, the Erlang VM is tailored to the semantics of Erlang and fault-tolerant systems, which allows it to be highly optimized for these specific requirements. This focused approach enables the BEAM to excel in areas such as concurrency, fault tolerance, and process isolation.
As mentioned earlier, the BEAM offers strong fault tolerance and process isolation. When a specific library or process encounters an issue and panics, it doesn't bring the entire VM down. Instead, only the affected process is terminated, allowing the rest of the system to continue functioning as expected. This level of isolation is achieved through BEAM's lightweight processes, which have their own heap and stack, and communicate with each other only through message passing.
Some features that the Erlang VM has, which are specifically tailored to fault-tolerant systems and not found in the JVM, include:
a) Per-process garbage collection: BEAM uses a per-process garbage collection approach, which enables it to efficiently handle millions of lightweight processes without the need for stop-the-world GC pauses.
b) Preemptive scheduling: BEAM's preemptive scheduler ensures that no single process can monopolize the CPU, and that all processes receive a fair share of processing time. This is a crucial feature for building highly concurrent and fault-tolerant systems. (The BEAM is ostensible an OS and can even be run bare metal!)
c) Supervision trees: The Erlang/OTP platform provides a built-in supervision tree mechanism, which allows developers to define strategies for monitoring and restarting failed processes automatically. This contributes to the overall stability and reliability of the system.
The JVM has made significant advancements in performance and garbage collection, the BEAM has been specifically optimized for the unique requirements of Erlang and fault-tolerant systems. Its focused approach allows it to offer features and optimizations that are particularly suited for these use cases.
In response to:
> But BEAM simply doesn't have anywhere near the level of investment required to be a leading platform. I wish there was more demand for it, but there isn't.
While it's true that BEAM may not have received the same level of investment as the JVM, it's important to note that the level of investment doesn't necessarily determine a platform's effectiveness or suitability for specific use cases. The Erlang VM has been designed and optimized over 35 years for the unique requirements of Erlang and fault-tolerant systems, which has resulted in a set of features and optimizations that cater specifically to these needs, its unparalleled in this field, and As such, the BEAM's focused approach allows it to be a strong contender in its niche, particularly for building highly concurrent, fault-tolerant, and distributed systems. It's not necessarily about competing with the JVM across the board; instead, it's about excelling in specific areas where the Erlang VM's unique features and optimizations provide tangible benefits.
Moreover, the Erlang/OTP ecosystem has a dedicated and passionate community that continues to drive its development and innovation. This community's efforts have ensured that BEAM remains a powerful and relevant platform for building resilient systems, despite having comparatively less investment than other platforms like the JVM.
OpenJDK's new GC doesn't require STW pauses, the JDK offers preemptive scheduling for millions of threads as well as supervision trees -- although, to be fair, these are recent changes that have, at least in part, been inspired by Erlang. All that is on top of one of the world's most capable compilers. I have long admired Erlang and its passionate community as well as how much they've been able to accomplish (and, as a VM engineer, studied its design), but between passion and being an actual top-tier competitor there's a certain reality gap.
Great, so you guys can spend the next 30 years writing OTP to work seamlessly and flawlessly on top of the new JVM with the same (memory, security, reliability, fault tolerance, etc) guarantees OTP has on the BEAM VM today :)