so what's the point? we can already write blocking code. The reason why people want to use reactive frameworks is because they don't want to block threads just because some I/O is happening.
Point is exactly you can block because blocking is cheap with virtual thread and can have 100's of thousand block calls with exhausting all system resources. VT takes few KBs of memory instead of few MBs of memory, so plentiful of VT is a feature. With this one can write plain blocking code get benefit of async scalability but with simple APIs structure and debuggable sync code.
Nima uses virtual threads where blocking call blocks only the current virtual thread but not the underlying carrier-thread, which can run another virtual thread while waiting i/o to complete.
But in this example, the entire code is still blocking because it has to wait for one call of "callRemote" to complete while waiting for the next one to be executed. So I don't get the point.
for (int i = 0; i < count; i++) {
resp.add(callRemote(client));
}
Before virtual threads, Java server would reserve a full OS thread while that code is being processed. OS threads use lots of memory and there might a limited number available. With a single incoming request or low traffic that is not a problem, but with many parallel requests the server would choke on using too much memory or OS threads.
Javascript moved on from callbacks to async/await, which colors the async functions and requires the programmer to be aware of the issues related to async/await.
For Java Project Loom takes a different approach. For example with server applications, the server would start running the code in a virtual thread and most of the code that looks like blocking will actually work like unblocking code. You get to use regular blocking code but you get async performance.
When the code needs to wait for a response, JVM can unmount the virtual thread from the OS thread and then the JVM can run another virtual thread in the OS thread. Once the response is returned and the OS thread is free, JVM can continue executing the original request.
There are some caveats to virtual threads. There is some overhead (but not a lot), some IO calls still block (but you don't get an indication about using blocking IO) and you might stumble on new kinds of bugs related to resource exhaustion (try reserving a million OS file handles).
But no more async/await, reactive programming, callbacks. Mostly just regular imperative blocking-looking code. I'm very excited about that.
This could be solved by using for..of in an asnyc function though.
It might look like surprsising behavior maybe, but putting the async keyword outside and looping without non-awaited callbacks is the only possible way to keep the semantics sound, no?
In other words, forEach would have to return a Promise too.
That's where the function coloring rightfully annoys programmers to catch their mistakes, there are if cause also lint rules for this.
AFAIK there are two active proposals that aim to provide this natively (and additonaly enable working with lazy and/or infinite iterators or generator functions):
AsyncIterator and
tc39/proposal-async-iterator-helpers
Things don’t have to be that way. You can reject the use of async/await in contexts that aren’t aware of it (delegating it to special APIs that properly handle it).
var remoteResultA = callRemoteA(client);
var remoteResultB = callRemoteB(f(remoteResultA));
Where handling a request means you have to make two http calls and those two calls have some order dependence. I.E. the result of one is used to construct the call to get the other.
What virtual threads give is the ability to write the code like I did above and still get ideal performance, as opposed to.
callRemoteA(client)
.then(remoteResultA -> callRemoteB(f(remoteResultA)))
.then(... your world lives here now ...);
Virtual threads are, from a language semantics point of view, the same as regular threads. Virtual threads map maybe 10000 "logical threads" to a handful (maybe 8 or so) actual operating system threads.
The big difference is that for each regular Thread you have exactly one backing Operating System thread.
I agree that this would be a better example that would make more sense (and I also think that the reasoning you provided should be added as context to that code snippet).
That entire section is running on a virtual thread, which is scheduled on a platform (OS) thread. While it blocks for each callRemote invocation the platform thread is free to process other virtual threads.
In addition, this implementation detail might help in gaining some insight: "The synchronous networking Java APIs, when run in a virtual thread, switch the underlying native socket into non-blocking mode. When an I/O operation invoked from Java code does not complete immediately (the native socket returns EAGAIN - “not ready” / “would block”), the underlying native socket is registered with a JVM-wide event notification mechanism (a Poller), and the virtual thread is parked. When the underlying I/O operation is ready (an event arrives at the Poller), the virtual thread is unparked and the underlying socket operation is retried."
Yeah but if for some reason, no other (or few other) request is coming in concurrently, then the CPU will just sit by idly. I don't understand why you wouldn't issue the "callRemote" calls concurrently, that seems like it's missing the point of writing non-blocking code.
Exactly. That's the point. It is beneficial only when you have a huge number of requests relative to the number of platform threads.
The idea is that you can map a million of these virtual threads on to a small number of platform threads, and the JVM will schedule the work for you, to achieve maximum throughput without needing to write any complicated code. Virtual threads are for high throughput on concurrent blocking requests.
EDIT: the sequential calls to "callRemote" still process in sequence, blocking on each call. Don't get confused there. But the overall HTTP request itself is running on a virtual thread and does not block other HTTP requests while waiting for the callRemote invocations to complete.
> EDIT: the sequential calls to "callRemote" still process in sequence, blocking on each call. Don't get confused there. But the overall HTTP request itself is running on a virtual thread and does not block other HTTP requests while waiting for the callRemote invocations to complete.
Yeah I got that. But then the use case seems to be much narrower, because not every scenario is one where many concurrent requests come in at the same time. If you write non-blocking code you'll get high throughput no matter the number of requests. Ok, maybe non-blocking code is harder to write (and I don't think it's actually that hard if you use, say, Kotlin coroutines), but honestly this seems to me like something that developers should learn eventually anyway.
Or maybe it's me being weird. But I remember 10 years ago when node.js was being hyped for being "fast" because it was "async" and now suddenly using threads and blocking operations is again all the rage now.
You certainly could make the callRemote calls in parallel, and there are easy and safe ways to do that (good old CompletableFuture or the new structured concurrency stuff). But doing that or not is completely independent of using Nima, so I think here they're just showing some very simple code, because this is an example of using Nima, not an example of writing concurrent code in general.
Because concurrent calls are confusing and create bugs.
The point this train wreck of an example is trying to make (and failing) is that it's fine to write sequential blocking code in your request handlers. They can take as long as they want ... seconds, minutes, days .... because the handler threads are now an infinite resource as far as the JVM is concerned.