[Spivak]

So loom uses interesting terminology when talking about this. They say that they’re preemptive and not cooperative because there’s not an explicit await/yield keyword that you call from your code but that isn’t the whole story because threads are only preempted when they perform IO or are synchronized. So you as an author can’t know for sure where the yield points are and aren’t supposed to rely on them but they’re still there. You’re not going to be forcefully preempted in the middle of number crunching.

I think most people would consider this a surprising notion of preemption where it’s out of your control-ish but also not arbitrary like it is for OS threads which still leads to basically the same problems and constraints as cooperative threads.

[cogman10]

Yeah... this is a place where I disagree with how the Loom devs define "preemptive". They are basically defining it as "most tasks will give up control when they hit a blocking operation". Yet, it's been my understanding that preemption means the scheduler can stop a currently operating task from running and switch to something else. That's not what happens with loom.

[pron]

> So loom uses interesting terminology when talking about this.

That is a common terminology. Wikipedia says: [1]

The term preemptive multitasking is used to distinguish a multitasking operating system, which permits preemption of tasks, from a cooperative multitasking system wherein processes or tasks must be explicitly programmed to yield when they do not need system resources. ... The term "preemptive multitasking" is sometimes mistakenly used when the intended meaning is more specific, referring instead to the class of scheduling policies known as time-shared scheduling, or time-sharing.

> threads are only preempted when they perform IO or are synchronized

First, they can be preempted by any call, explicit or implicit, to the runtime (or any library, for that matter). For all you know, class loading or even Math.sin might include a scheduling point (although that is unlikely as that's a compiler intrinsic). We make no promises on when scheduling can occur. Not only do threads not explicitly yield, code cannot statically determine where scheduling might occur; I don't believe anyone can consider this "cooperative."

Second, Loom's virtual threads can also be forcibly preempted by the scheduler at any safepoint to implement time sharing. Currently, this capability isn't exposed because we're yet to find a use-case for it (other than one special case that we want to address, but isn't urgent). If you believe you have one, please send it to the loom-dev mailing list.

The reason it's hard to find good use cases for time slicing is as follows:

1. If you have only a small number of threads that are frequently CPU bound. In that case, just make them platform threads and use the OS scheduler. Loom makes it easy to choose which implementation you want for each thread.

2. If you have a great many threads, each of which can infrequently become CPU-bound, then the scheduler takes care of that with work-stealing and other scheduling techniques.

3. If you have a great many threads, each of which is frequently CPU-bound, then your cores are oversubscribed by orders of magnitude -- recall that we're talking about hundreds of thousands or possibly millions of threads -- and no scheduling strategy can help you.

It's possible that there could arise real-world situations where infrequent CPU-boundedness might affect responsiveness, but we'll want to see such cases before deciding to expose the mechanism. Even OSes don't like relying on time-sharing (it happens less frequently than people think on well-tuned servers), and putting that capability in the hands of programmers is an attractive nuisance that will more likely cause a degradation in performance.

[1]: https://en.wikipedia.org/wiki/Preemption_(computing)#Preempt...

[lmm]

Isn't the point of preemption to degrade gracefully when the cores are oversubscribed? E.g. the first system I worked on ran potentially CPU-heavy work from various clients, and used per-client threads to isolate them; every so often clients would find ways to get their thread stuck doing a large amount of CPU work (e.g. regex backtracking) and although these were in some sense bugs (and we did fix them), it was very useful that even if one or two clients blocked all their threads (which was often more than our number of physical cores), this wouldn't completely block other clients' threads from running.

[pron]

There's no doubt forced preemption could help, but I'm still unsure about what the right algorithm is; probably not time sharing.

Suppose you have 100K threads, and only 1% of them become CPU-bound for 100ms. That could take down your 32-core server for 3 seconds, which is bad. But suppose we had 10ms time-slices. Then, those busy threads' latency might go from 100ms to as high as a few minutes, which means effectively taking them down. The scale has a qualitative effect here. So, rather than time-sharing, it might be better to optionally install some other preemption policy -- maybe something that indefinitely suspends threads that behave badly too often and puts them in some collection.

The point is that time-slicing will probably not be helpful in sufficiently many cases, and we don't yet know what will. We'd like to gather more data before offering something. In some other languages/runtimes it might be worthwhile to just expose a capability and see what people do with it, but with Java, within five minutes you'll have twenty libraries doing time-sharing, and thousands of people using them blindly whether it's good or bad for them (just because they say they do time-sharing, and that's good, no?), and now there's just noise and bad habits everywhere. This is nanny-state governance, but we've learned our lesson, and you can't be too careful with an ecosystem this big.

[lmm]

Sure. Ultimately you've got the same problem as an OS scheduler and recognising whether threads are CPU-bound or IO-bound and treating them separately is probably going to be part of that.

I appreciate not wanting to do things until you can do them right, but equally if you advertise this as a preemptive runtime, people are going to expect that they can use it to throw 32 CPU-spinning threads onto 8 cores and have it behave gracefully. It sounds like from a user's point of view on day 1 this runtime will be the worst of both worlds - you need to take care to not do big chunks of CPU work without yielding, but you don't get the full control that a traditional "userspace" cooperative multitasking framework would give you.

[pron]

But the experience people have already been having with the Early Access is overwhelmingly positive. Even without forced preemption, "preemptive" is far less misleading than cooperative, even considering the common confusion between preemptive scheduling and time-sharing.

While OS threads might indeed handle 32 spinning threads on an 8-core machine more gracefully, switching between implementations of threads is easy so such a "mistake" is inconsequential, and no OS handles 320,000 spinning threads gracefully, and people know that that is the scale of threads that virtual threads exist to serve.

[lmm]

You're right that calling it "cooperative" would be worse. Still, I suspect Early Access users are paying a lot more attention to the details (and are more knowledgeable users in general) than GA users will; switching thread implementation might be "easy", but I suspect most users will want to use Loom without tuning anything at all. So safe defaults are very important (and I'd suggest that for the default config, safely handling 32 spinning threads on 8 cores is more important than handling 320,000 mostly-sleeping IO-bound threads).

Which is not to say I have a better idea (other than "make the defaults magically do everything right", which is obviously hard).