Hacker News new | ask | show | jobs
by willvarfar 5251 days ago
I happen to be using Python's built-in new multiprocessing module. It has a suitable queue. I am using the same logic to talk to rabbitmq. I would avoid tcp.
1 comments

Where's rabbitmq in your architecture? What's your concern with TCP?
I meant I I'd rather use Queue.Queue or multiprocessing.Queue to talk to a worker than TCP. Its just my preference.

Its generally the principle of least-power is that you'd use a thread (nice for the callback handling, and for 'interactions') or a process (if you worry about the GIL, but it complicates callbacks and interactions), and only jump to a TCP server if you really really could justify it.

I also meant that the approach of workers that do coalescing is useful for non-DB things too; we're using a worker to buffer rabbitmq, for example.