|
|
|
|
|
by guiriduro
2863 days ago
|
|
If your tasks are fairly coarse-grained (take >50ms each),
Celery [1] has existed for a several years; takes a bit of setting up but works well, its very flexible. If your needs are simple, don't forget that your common or garden webserver can parallelize workloads too (distribute web requests to workers on multiple cores), it depends mostly on your client code for fan-out, and redis has worked well for synchronization for me. Nowadays you can also use serverless to parallelize coarse-grained workloads in the cloud. [1] http://www.celeryproject.org/ |
|