Hacker News new | ask | show | jobs
by dec0dedab0de 3901 days ago
That really depends on your requirements. If you need multithreading (not multiprocessing), you cannot use Python.

About to show my ignorance, but when is multithreading useful when multiprocessing is not? Assuming it is a use case that is suited for a high level dynamic language to begin with.

2 comments

Multithreading lets you share memory. Multiprocessing requires you to serialize your data and copy it to the other processes. That can be very expensive.

This only matters for performance-critical applications.

Let's say you have a REST request come in. In order to fulfill that request you need to make 10 REST requests of your own to various back-end systems. If you can make some of those requests asynchronously you'll greatly reduce your response time. While I suppose you could mangle your way through it multi-process, it seems like a bastardization of the model.
If you're just sending HTTP requests, regular threading in Python works fine - waiting for a socket response doesn't block the execution of other threads.

Python threads are real threads, and things like blocking socket IO does not block Python execution in other threads.

While I suppose you could mangle your way through it multi-process, it seems like a bastardization of the model.

That kind of sounds like you're nitpicking over an implementation detail. Which brings me back to my original confusion. If I have an interface that allows me to articulate concurrent tasks, I don't see the difference between threads, processes, or separate machines.

Again, this is assuming we're not in a resource constrained environment, and we're not doing something so processor heavy that it really should be compiled.

Think I'd rather use asyncio in this case. Assuming most recent Python might not be fair though I guess.
It looks like a job for an event loop (tornado/twisted/asyncio), not for a thread pool.