Hacker News new | ask | show | jobs
by ditados 4741 days ago
no mention of gevent, celery, etc. As someone who runs thousands of concurrent tasks in a mix of process/gevent (one UNIX process for each 100 greenlets across 48 cores on two boxes), I find the OP's toliling rather misguided.
1 comments

The author's use case is clearly very different from yours. He is talking about CPU-bound processes which need to share a large amount of memory with each other. In this case, multiprocessing and message passing is not really the best fit. Multithreading or shared memory results in far less CPU usage and memory duplication.