[wormlord]

Oh man, I wish this was a bit more mature. I am writing a distributed image processing app right now with Celery, Flask, and SQLAlchemy, but I am finding it difficult to KISS. I was tempted to use Django but couldn't justify the overhead. If they get this cleaned up in the next year that would be awesome!

[Paul-Craft]

One thing I have learned after nearly a decade in the industry is that you should just never, ever, ever use `celery`. It's overly complicated to program with; the monitoring tools are not great; and the one guarantee you can make is that at some point, you will have some kind of difficult to debug, mysterious failure involving it.

If I needed asynchronous background processing in Python today, I'd definitely be looking into `django-tasks.` If you asked me yesterday, I'd have told you I'd use SQS / whatever the platform equivalent was, or, failing that, I'd probably go with `dramatiq`.

Edit: LOL, I can't believe I forgot to mention what a fscking nightmare `celery` is to deploy and configure. UGH.

[tecleandor]

Which django-tasks? This one that started development a month ago?

https://github.com/RealOrangeOne/django-tasks

[Paul-Craft]

The very same. As mentioned in the article, it's going to be distributed with the newest version of Django.

[tecleandor]

I don't understand that. The article says '...an existing external project that served as a proof of concept', and the DEP calls it '...an "early-access" demo to get initial feedback'.

Although it looks interesting and I'll keep an eye on it, if I needed something right now, I wouldn't use it.

[wormlord]

Does Django-tasks support different workers with different environments? Say I need a worker with GPU capabilities, can it do that?

[Paul-Craft]

I don't have any idea. Take a look here: https://github.com/RealOrangeOne/django-tasks

[dacryn]

justify the overhead? But you are using flask and sqlalchemy?

You made the wrong choice I would guess. Django does not have 'overhead' to justify in that scenario.

[Paul-Craft]

Here's how much is installed when you do `pip install django` currently:

    122370  django          python=103934,javascript=18407,xml=29
    2734    sqlparse        python=2734
    1110    asgiref         python=1110

This will probably change when `django-tasks` gets merged in.

Here's what installing `flask` + `sqlalchemy` pulls in:

    141314  sqlalchemy      python=141314
    11464   werkzeug        python=11159,javascript=305
    8934    greenlet        cpp=4993,python=2729,ansic=1092,asm=120
    8606    jinja2          python=8606
    5842    click           python=5842
    3907    flask           python=3907
    2036    top_dir         python=2036
    644     itsdangerous    python=644
    487     markupsafe      ansic=278,python=209
    349     blinker         python=349

Looks like the same order of magnitude to me, which is surprising. I had always thought of `django` as being heavy weight and opinionated, while `flask` + `sqlalchemy` was light and modular.

All numbers generated using David A. Wheeler's 'SLOCCount'. :-)

[evantbyrne]

I suspect that people come to the conclusion that Django has performance issues by looking at synthetic benchmark rankings between frameworks. In that sense it may or may not be relatively slow, but it is probably fast-enough for almost all web services in-practice. And if you measure with all of the speediest HTTP interfaces and find out it isn't fast-enough, then frankly Python is probably not the right choice at all.

[Paul-Craft]

I've never heard of anybody complaining about Django per se having performance issues. When people say Django is "heavy weight," I tend to think of how it bundles a lot of stuff for you, which can make it easy to build an app; while, at the same time, if you end up deciding you want to step slightly off the garden path and do things juussssst a little differently than Django wants you to, you're probably gonna have a bad time.

OTOH, Flask is too small to impose much of a conceptual framework on you, and SQLAlchemy generally feels like a fairly thin layer of Python syntactic sugar over top of SQL. When I write queries in SQLAlchemy, it seems like they always tend to come out looking more like what I would write in plain SQL than queries in Django do. Granted, both ORMs can start doing things like joining tables behind your back, which can certainly cause performance problems, but that's a problem that's common to both, and not just Django. ¯\_(ツ)_/¯

[wormlord]

> if you end up deciding you want to step slightly off the garden path and do things juussssst a little differently than Django wants you to, you're probably gonna have a bad time.

Yep! I am using Flask + SQLAlchemy so I can have an orchestration layer that handles all the business logic, and just takes in my Flask app, Celery app, and database sessions as dependencies. I don't know if there's an equivalent way to do that in Django. If there is, it seems like it would be more trouble than it is worth.

[wormlord]

By overhead I didn't mean performance, just all of the extra configuration stuff that comes with Django. Although I'm still open to moving to Django as this project grows. A big benefit of a framework like Django is that everything is already organized for you, I kind of feel like I am re-inventing the wheel a bit here.

I am keeping the business logic separated out though so I can easily move to Django if needed.