[Spivak]

> On asynchronous protocols like ASGI, despite the fact the concurrency model doesn't change that much – we shift from one event loop per process, to one event loop per thread – just the fact we no longer need to scale memory allocations just to use more CPU is a massive improvement.

It's nice that someone else recognizes that event loop per thread is the way. I swear if you said this online any time in the past few years people looked at you like you insulted their mother. It's so much easier to manage even before the performance improvements.

[yupyupyups]

No, because you can't kill a Python thread, but you can kill a process. That is a significant limitation to think about, especially when executing something long-running and CPU intensive such as large scientific computations.

If your thread gets stuck, the only recourse you will have is to kill the entire parent process.

[hunterpayne]

Wow, I seriously question the quality of any project where this is a consideration. I would also strongly recommend you hire some better devs and rewrite that project in another language with better concurrency features. Java (or anything on the JVM) would lead that list but plenty of languages would be suitable.

Also, sharing memory between processes is very very very slow compared to sharing memory between threads.

[yupyupyups]

The assumption is that they never share memory.

Any time you have an arbitrary independent task that can be started and then stopped by the user, you will need processes in Python.

Java is a lot better at concurrency and has a vast library for concurrency primitives like atomic operations (CAS etc.)

Python's strengths lies in its strong ecosystem and ease of use. Many times that overshadows the benefits of Java's competent concurrency features.

[tommmlij]

That plus you can have memory leaks when you run heavy stuff in threads...

[jeremyjh]

If I worked on a project this shitty I sure wouldn't be telling people about it in public. Just the fact that you think this is advice other people need to hear is telling.

[yupyupyups]

What advice? I'm not sure what you mean.

Do you mean that there is no valid use-case where you wish to interrupt a blocking thread due to I/O, for example?

Or maybe you think Python can do that. Maybe I'm wrong, but as far as I can tell Python is not good at doing that.

Perhaps my choice of word "kill" confused you, and in that case I should've picked better wording reflecting what I meant. Perhaps you thought I meant people should kill threads/processes instead of fixing buggy CPU-intensive tasks? That's certainly not what I meant.

You don't seem to be very charitable in how you interpret what I'm saying. Instead of a very vague comment, it would've been better if you'd have explained your concern, and given me a chance to understand you.

[int_19h]

> Do you mean that there is no valid use-case where you wish to interrupt a blocking thread due to I/O, for example?

This is certainly a common case, but killing a thread with a blocking native call on it is a very poor way to do so (not the least because you don't know what locks it might be holding at the moment it gets killed - imagine what happens if that's one of the locks used by low-level heap, for example). The proper way to address it is to use asynchronous I/O APIs that allow for cooperative cancellation. Unfortunately Linux doesn't exactly have a good track record in that department, which is why people do these kinds of hacks. On Windows you get stuff like https://learn.microsoft.com/en-us/windows/win32/fileio/cance....

[yupyupyups]

I shouldn't have used the word "kill" in the context of talking about threads, that seems to have created some misunderstandings.

>The proper way to address it is to use asynchronous I/O APIs that allow for cooperative cancellation.

I completely agree here. Async is the go-to for IO-bound operations and the ability to cancel (sends an exception to the task) is a very useful feature.

Killing threads, as in non-gracefully stopping them is a bad idea regardless of language, and not something I would encourage nor something I do myself.

In Python, if there is CPU-bound long-running routine that need to be executed concurrently, then this can be done using multiprocessing. I'd say, if external resource usage is well-defined then a good way to stop such a task would be to send a sigterm, wait, then send a signal 9 if it hasn't stopped after a grace period. If needed, perform a cleanup afterwards.

The problem with python and threads in my experience is that even a graceful interruption of individual threads can be tedious to get right.

Thanks for the link btw.