[asksol]

Celery was never meant as a replacement for cron, it was simply a nice bonus that fits the messaging pattern well. Writing a task queue is actually very simple using for example Redis, but that doesn't necessarily mean Celery is over-engineered IMHO. It's very easy to forget the support required once your system is in production.

Disclaimer: I'm a contributor

[k_sze]

By "contributor" you mean you are the main author, right? XD

Joking aside, we use Celery in production for asynchronous number crunching. We build a web UI in Django that raises Celery tasks that run long number crunching jobs. When the number crunching is done we look at the report through our Django web UI.

It works well enough, though we wish there were better built-in task management. E.g. A built-in API to reshuffle tasks (e.g. for I/O resource balancing) would be nice. celery purge also seems too drastic in purging every task in every queue. Or maybe we're just doing it wrong.

[andybak]

I'm not saying Celery is over-engineered in general. It's just over-engineered in the context I've often seen it recommended in. i.e. for people learning or people with fairly modest requirements.

[collyw]

Yep, I first asked on Stack overflow the best way to achieve a background task in a Django app, and the answers all said Celery. Considering it got run around once a week, it was a very over engineered solution I ended up with.

Its useful to know Celery, and gets used in a proper context in my current work, so I guess learning it wasn't a waste.