Hacker News new | ask | show | jobs
by tptacek 6332 days ago
Great article.

Background/deferred job processing has been immensely painful for us, and I have no idea what the accepted best job queue is. We're still stuck on backgroundrb, which is a nightmare. Nanite looks like overkill, and has too many deps.

7 comments

Workling has worked well for us -- Rany Keddo is a phenomenal open source guy.

Tobi's deferred jobs also looked totally solid to me. That one's proven because it drives Shopify. We use his liquid plugin, which is also stellar.

That underlines a big issue with Rails dev even today -- its really hard to know what's good / what works, and what is just some weekend project for someone.

Delayed-Job works great for Lighthouse/Tender from us, and for Github too. It's definitely a great starter queue until your needs necessitate something more heavy duty like nanite.
Found Tobi's delayed_job (not deferred) --- I think I like the design better than Workling; fewer moving parts, just a database backend end a rake script. The biggest problem we have with backgroundrb (which again: nightmare) is not really ever knowing the state of current running jobs.
yeah backgroundrb sucks, who wrote that shit? (me and i wish i never did)
(oh, and thank you).
We're currently using ActiveMQ with ActiveMessaging as the Rails bindings. One reason for this was that we wanted something which would be fairly language/platform neutral if in the future we decided that Ruby/Rails wasn't the right choice for certain bits of the app.

RabbitMQ is another option to consider in this area.

We simply use an in-memory queue. Beanstalkd has worked quite well for us. (Not a rails app though)
Beanstalkd + async_observer (Rails plugin) have worked well for me. Add a Munin script to monitor the queue size (http://gist.github.com/36116) and it's a pretty effective and manageable system.
Huh, I've been quite happy with backgroundrb lately, both for repeating (cron-style) background work and for deferred processing. We've had little trouble tracking worker status once we switched to the memcached-based result cache, which allows the app to interrogate workers however it sees fit...
I'm 'stuck' on backgroundrb too. Until now I thought it was the default and best way to run processes in the b/g for a Rails site.

I'm eager to see if anyone else can recommend alternatives that they've had good experiences with.

We're using Background Job (bj) quite happily, though we've been told that Delayed Job (dj) is the new hip thing to use in its stead.
the async plugin is quite nice. you can replace

>> myobject.very_slow_method

with

>> myobject.async :very_slow_method

and it's executed in background

http://github.com/lassej/async/tree/master