[chewbranca]

I keep seeing these (admittedly cool) redis based job queues popping up for different languages, such as resque, pyres, now kue, etc, however I'm disappointed to not see any standardization among them. Obviously they're different projects created by different people, but it would be great to have a common protocol amongst them, especially since they're all just json based messages.

I want to be able to use a job queue as a standard way to communicate between various language backends, so I can use different tools for different jobs. I've started looking into beanstalkd as it has an impressive list of client libraries: https://github.com/kr/beanstalkd/wiki/Client-Libraries.

I'm curious to hear if anyone has any thoughts/experiences/recommendations on using open source job queues to facilitate background processing between different languages and backends. I'm already using redis and resque, something like rabbitmq is overkill, and I just started looking into beanstalkd.

[technoweenie]

coffee-resque and resque work together really well. I didn't mess with the python resque lib much, but I was able to get it to read jobs queued by ruby. With a little work, resque could work as a cross platform queue if you wanted. The main issue I ran into is the way the libs bind resque jobs to objects. For instance, the ruby resque library queues jobs by the ruby class name (Foo::Jobs::Bar), which doesn't translate well to python or node.

A good first step would be defining a spec based on how resque or kue work so that alternate language ports can be written. In the case of resque, other interfaces like its failure or plugin system should also be documented, so that those libs could be ported too.

[tjholowaychuk]

i agree interface wise it would be interesting, though i think many have slightly different goals and techniques, and frankly they are really basic, but UI-wise it could be neat