[atomicnumber3]

The main problem is that it's not a matter of "speed" but just of congestion.

If you write a program using blocking IO and Platform (OS) threads, you're essentially limited to a couple hundred concurrent tasks, or however many threads your particular linux kernel + hardware setup can context switch between before latency starts suffering. So it's slow not because Java is slow, but because kernel threads are heavyweight and you can't just make a trillion of them just for them to be blocking waiting on IO.

If you use async approaches, your programming model suffers, but now you're multiplexing millions of tasks over a small number of platform threads of execution still without even straining the kernel's switching. You've essentially moved from kernel scheduling to user-mode scheduling by writing async code.

Virtual threads is a response to this, saying "what if you can eat your cake and have it, too?" by "simply" providing a user-mode-scheduled thread implementation. They took the general strategy that async programming was employing and "hoisted" it up a couple levels of abstraction, to be "behind" the threading model. Now you have all the benefits of just blocking the thread without all the problems that come from trying to have a ton of platform threads that will choke the linux kernel out.

[riku_iki]

> couple hundred concurrent tasks, however many threads your particular linux kernel + hardware setup can context switch between before latency starts suffering.

and there any benchmarks saying it is couple hundred threads, and not 100k threads?..

couple hundred is about thread per core on modern CPUs..

Also, my belief is that JVM itself adds lots of overhead.

[ye-olde-sysrq]

>Also, my belief is that JVM itself adds lots of overhead.

The JVM has ~nothing to do with the scheduling of platform threads

>and there any benchmarks saying it is couple hundred threads, and not 100k threads?..

It depends greatly on your hardware

>couple hundred is about thread per core on modern CPUs..

It depends on the hardware, of course

Overall, yes, you could probably run a lot of concurrent processes on a c7i.48xlarge in 2023. But why would you want to do that when, if you used virtual threads (or async), you could do the same work on an c7i.large? That's the whole point of this. There's no reason to waste CPU and memory on kernel context switching overhead.

[riku_iki]

> The JVM has ~nothing to do with the scheduling of platform threads

I didn't say it schedules platform threads, but JVM has many other performance issues with its GC/object identity/lock monitoring model, which makes it harder choice for ultra-high performance apps, so virtual thread scheduling may improve your app perf by 5%, where other 95% stuck in other bottlenecks

> But why would you want to do that when, if you used virtual threads

because virtual threads are not supported in ecosystem well, and it will take another 10 years for it to catch up, before that most API/libs will use old concepts and you take large risks of fragmentation by introducing virtual threads to your app.

Also, you can avoid all switching with 20 years old ExecutorService, where you have exactly the same m:n mapping between tasks and machine threads.

[ye-olde-sysrq]

>I didn't say it schedules platform threads, but JVM has many other performance issues with its GC/object identity/lock monitoring model, which makes it harder choice for ultra-high performance apps, so virtual thread scheduling may improve your app perf by 5%, where other 95% stuck in other bottlenecks

Switching from async to virtual threads is typically not an improvement to performance in well-factored async code. The primary benefit is a clearer programming model that's significantly easier to get right and less code to implement while still the same performance.

>Also, you can avoid all switching with 20 years old ExecutorService, where you have exactly the same m:n mapping between tasks and machine threads.

You misunderstand the point of virtual threads. There is no need to pool them, as (most) executor service implementations do. And the whole point is that you no longer need to avoid blocking them, which pretty much every currently-existing solution will be doing. There's 0 benefit to switching without also updating your code to take advantage of the new paradigm.

>because virtual threads are not supported in ecosystem well, and it will take another 10 years for it to catch up, before that most API/libs will use old concepts and you take large risks of fragmentation by introducing virtual threads to your app.

What risks? The whole point of virtual threads vs async/await is that they don't color functions in the same way that async does.

Also, pretty much every notable framework already has support for virtual threads (Spring, jetty, helidon, and many others). The API has been the same for a couple years at this point.

[riku_iki]

sorry, I disagree with most of your points but not interested in continuing discussion.

[kaba0]

You can just swap a single line in your ExecutorService then for all the benefits.

[riku_iki]

ExecutorService can be configured in many different ways, for example I often create one with boubned queue and CallerRunsPolicy, I somehow couldn't find equivalent virtual executor.