[mirsadm]

>For the second type of system, I can't remember the last time where I wrote one where I had in-process parallelism/concurrency with shared resources (even in langs where there's no GIL). It would just get too confusing and hard to reason about. Any optimizations were mostly based on intelligent batching. For parallelism, you'd just have multiple _completely_ independent processes, probably across multiple machines.

Interestingly I'm working on something like this right now and do have large shared resources which meant I had to abandon using a multiprocess strategy.

I don't see why it would be confusing though, provided the shared resources are read only.

[nullspace]

For such applications, isn’t the parallelism count usually static and limited? I think there will be good benefits for distributed system frameworks for python. Id agree.

A couple of years ago, I implemented in process parallelism for a system I was maintaining at $JOB. I was happy the system was in Go and not Python. But it was an exception to the rule in my experience.