[sifoobar]

Having recently been down this particular rabbit hole myself; I just want to note that there are other possible strategies, a GIL is not the only alternative to a fully concurrent runtime.

My own baby, Snigl [0]; doesn't even support preemptive threads, only cooperative multitasking; with a twist, since blocking operations are delegated to a background thread pool while yielding when multitasking.

[0] https://gitlab.com/sifoo/snigl#tasks

[amelius]

That is a nice strategy but it only allows IO to be parallel, and leaves the CPU sequential. Programs may be concurrent (because of the cooperative multitasking) but not parallel (because there is only one CPU thread). Users may find it disappointing that they can only use a single core.

Also, keep in mind that cooperative multitasking may cause unexpected high latencies, which is unfortunate e.g. in GUI applications and web servers; this is a result of queueing theory, and an example is given here: https://www.johndcook.com/blog/2008/10/21/what-happens-when-...

By the way, on POSIX systems there is a way to schedule background IO operations without even using threads: http://man7.org/linux/man-pages/man7/aio.7.html

[sifoobar]

I am aware, it's simply the most portable universal solution to the problem that I could think of. AIO comes with its own share of issues; Unix (and thus Posix) are pretty much dead; and it's not universal, I can't use it to open a database connection unless the database has support built-in.

From what I know, cooperative multitasking suffers significantly less from unpredictable performance than preemptive threading. The biggest source of uncertainty in Snigl is the preemptive IO loop.

The things is that I really don't feel like writing a concurrent runtime; been there, done that. I'm planning something along the lines of Erlang's processes and channels based on separate interpreters for those scenarios.

[amelius]

Ok. A problem with separate processes is that you have to serialize your data when passing messages. This is a pity because functional data structures + structural sharing can be very efficient. For example, I can't imagine how someone would implement a high performance database without structural sharing.

[sifoobar]

Processes by name, from Erlang.

They're implemented (in my mind so far) as preemptive threads, one per interpreter; which makes them slightly more heavy-weight than Erlang's NxM and a nice complement to purely cooperative multitasking.

[amelius]

Sounds interesting. I'm not familiar with Erlang, and I still wonder how shared memory is managed.

[sifoobar]

It's one of those languages that does things differently to solve actual issues, not to check boxes.

From my limited experience, Erlang doesn't share data between processes; you throw it over the fence by sending to the process inbox, which is where the locking takes place.

Still, shuffling data between OS threads is an easier problem to solve than serializing between OS processes.