[robto]

I've been meaning to try this out, from my read it's a declarative way to get some structured concurrency. I work in a codebase that heavily uses core.async channels to manage concurrency and you really need to pay close attention to error handling. When you're spawning new threads you need to set up your own custom machinery to re-throw errors on a chans, close chans, and it looks like core.async.flow is a way to do all of this declaratively.

Just like `core.async` itself was a frontrunner of Virtual Threads on the JVM, I view `core.async.flow` as the Clojure version of the upcoming [0]Structured Concurrency JEP. I do wonder if it will use that under the hood once it becomes stable, the same way `core.async` is planning to move away from the `go` macro to just have it dispatch a virtual thread.

[0]https://openjdk.org/jeps/453

[puredanger]

I don't think it would be feasible or wise to structure core.async to use Structured Concurrency, although Structured Concurrency is trying to tackle some of the same problems as flow but in a different way (more akin to data flow style concurrency).

[xmcqdpt2]

We've been looking at virtual threads in a project at work and what we found is that it is quite difficult to adapt existing code to run with virtual threads.

For example, class initialization pins a thread so any singleton defined in the standard, recommended Java way (using a static inner instance of an inner class) can hang your program forever if the singleton definition suspends. And because they worked really hard on making async colourless, there is no way to know that a method call will suspend. This is a known issue with a workaround if the suspend is due to a network call,

https://bugs.openjdk.org/browse/JDK-8355036

which is useful for some applications (web servers). Figuring out that this is why my program was hanging required quite a bit of work too. We are still frustratingly far from the ergonomics of Go concurrency (all threads are virtual threads, hangs automatically panic).

[puredanger]

Clojure's focus on immutable data and pure functions side-step a lot of the trickiest issues with virtual threads. It's often not hard to isolate the I/O parts of your program into flow processes at the edges that can be mapped to the :io pool using virtual threads.

[xmcqdpt2]

The trickiest problems with VT aren't due to mutability. Mutability is problematic with any kind of concurrent programs.

The difficult problems are execution problems like pinning. There are plenty of existing concurrency libraries on the JVM (Cats Effect, clojure async, Kotlin coroutines, RxJava, quarkus, etc etc). The promise of VT is that you will no longer need those for scheduling and execution of work (whether that's tasks, fibers, coroutines, actors etc.) This only works if you use VTs throughout, not just on IO pools.

[gf000]

> can hang your program forever if the singleton definition suspends

I am no expert on the topic, but this seems like a very edge case scenario, that is not trivial to reproduce on even the linked bug ticket. Do you think it really is that big of an issue?

[xmcqdpt2]

It's really not hard to reproduce,

- vt1 locks lock1

- vt1 suspends on lock2

- n VTs attempt to initialize a singleton that requires lock1, so they all suspend within pinning class init.

- you release lock2.

- all platform threads are pinned, so vt1 can't run and you hang forever.

There is no lock inversion and progress would have been entirely possible with platform threads, even with just one.

It happened on our system because we have parallel streams that all access the same singleton at the same time, which is fairly easy to do (you have lots of parallelism, you have a map operation that needs a value from your singleton and that's it.)

The solution is to never suspend while in a static block, but it's hard to generalize because... any method may suspend, and there is no way to know that it will because of colourlessness. And also the singleton pattern is common, often involve accessing expensive resources or IO (and suspending) and doing so with class init lock is recommended and common,

https://shipilev.net/blog/2014/safe-public-construction/#_sa...

In our case this involves scala's object {} which is a singleton defined using class init. Kotlin probably works the same way.

[gf000]

Thanks for the reply, I see!

Hopefully it will be solved, similarly to the other pinning issues then! Though wouldn't reserving a few platform threads solve the issue?