Do you mean a cronjob that calls a Django manager command to do the work? Or invokes an API method? From my experience cronjobs have a lot of downsides as well. They're great for doing tasks local to the server the job is running on. Not so great for the kinds of tasks (periodic or transactional) that Celery/a real queue is designed for.
Crons are fine for running Django commands that are just talking to the same services as your web views (databases, cache, email services etc). It should be fine to let them run in their dedicated VM/containers, using Ansible etc to keep the crontab configurations in the repo.
Celerybeat has the advantage of better visibility e.g. you can configure them in the Django admin and check when they are running. If you are not using Celery though and your needs are simple, it's easier to just use plain crons.
I use them for invoking django commands on the same server.
I do use celery for transactional jobs though. It’s only periodic jobs that get called with cron.
For the context, I do this on small web apps with less than 20 dedicated servers (real servers not VPS), so there is a “manager” server that does nothing but run periodic tasks, management interventions, and cleanup operations.
Yeah. Or a simple cron wrapper like django-cron to get the best of both worlds.
For background tasks - you can just spawn a background process and keep a simple status table in the db so the main app can check if it's completed (assuming you even need that)
And for task queues that can handle the traffic most sites will need there's things like django-huey.
> For background tasks - you can just spawn a background process and keep a simple status table in the db so the main app can check if it's completed (assuming you even need that)
I don't know if making my own bespoke queue system is a great idea. It seems simple enough, but it gets so much more complicated once you start seeing issues with it. Orphaned task processes sticking around on the server forever, concurrency control, error handling, etc. I'll pretty much always just use celery and not have to worry about it.
I'd never argue for building your own - just for using something simpler than Celery
It's not so bad now as CI/CD, Docker etc have made complex deployments easier to handle. But back when I was wrestling with Django simply deploying Celery on a new host could easily waste an afternoon and all those dependencies made me very nervous about the overall complexity.
I still weigh carefully anything that adds another long-running process or non-Python dependency to my sites.