Hacker News new | ask | show | jobs
by nonpme 4835 days ago
The biggest problem I have with Python frameworks is the fact that it's hard (at least for me) to configure server and just write aplications. I came from PHP background, where you install few packages, configure apache/nginx (which is quite fast) and things just work. I'm learning python for some time now, but I still can't create working python environment... I know django basics, and use its server for developing, but I don't know how to configure nginx on my VPS so I can have multiple python webapps (my simple apps, django apps, flask etc.). Someone has good resources for that (believe me, I read a lot of tutorials)?
11 comments

If the application in question is using WSGI to serve the content, I would e.g. advise to use nginx's built-in uwsgi protocol support (here 'uwsgi' denoting protocol and not the popular application server uWSGI itself) for performance reasons. I agree with sirn that uWSGI should not be easily overlooked. :)

uwsgi protocol is built-in in nginx >= 0.8.40; for older versions (and a general walkthrough), see http://uwsgi-docs.readthedocs.org/en/latest/Nginx.html

Here's the relevant bit for nginx's .conf:

  First of all copy the uwsgi_params file (available in the nginx directory of the uWSGI distribution [uWSGI here meaning the uWSGI application server, to be installed separately via your distro's package manager or e.g. pip]) into your Nginx configuration directory, then in a location directive in your Nginx configuration add:

    uwsgi_pass unix:///tmp/uwsgi.sock;
    include uwsgi_params;

  – or if you are using TCP sockets,

    uwsgi_pass 127.0.0.1:3031;
    include uwsgi_params;
(I would advise the former for performance reasons (unix sockets are faster))

  Then simply reload Nginx and you are ready to rock your uWSGI powered applications through Nginx.
Here are my nginx relevant directives - in server {} scope,

    set $server_root   /home/z/www/<basically,your_web_root>;
    set $uwsgi_socket  unix:$server_root/backend/var/uwsgi.sock;
Then (still in server {} scope),

    location /api { # my flask app serves <domain>.tld/api
      uwsgi_pass $uwsgi_socket;
      include uwsgi_params;
    }
That's it! That way I can reuse $uwsgi_socket as many times as is needed, etc.

The most simple way to launch the uWSGI app server would be to e.g.

  z@<hostname>:~/www/<web_root>/backend/lba$ cat test.sh

    #!/usr/bin/env bash
    uwsgi --post-buffering 1 --enable-threads -s /home/z/www/<web_root>/backend/var/uwsgi.sock -M -p 2 -w lba:app --chmod-socket 666 --pp ..

  uwsgi --help | grep "\-pp"
    --pp    add directory (or glob) to pythonpath
So we add a parent directory ("lba" containing the actual application files, e.g. __init__.py (can be empty) so that the directory could be imported (lba:app in uWSGI launch), etc.), tell uWSGI that this is to be the parent uwsgi process (-M), point it to its socket (-s), set number or processes (-p), tell it to actually load our 'WSGI module' (-w), and supply other per-application parameters which are not necessary if you just want to test things out.

app.py (lba:app) may simply contain (from http://flask.pocoo.org/docs/quickstart/) e.g.

  from flask import Flask
  app = Flask(__name__)

  @app.route('/')
  def hello_world():
    return 'Hello World!'

  if __name__ == '__main__':
    app.run()
That way you can test the sample app by simply sudo service nginx reload and ./test.sh (you might need sudo for the latter, too; also, might need to touch ../var/uwsgi.sock and look into uwsgi --help | grep "user") You can then put the test.sh (or the uwsgi call line itself) into e.g. supervisor - if the user privilege nuances are sorted out, you more or less have a production-worth setup (as far as nginx <-> uwsgi <-> your app is concerned), methinks.

edit sirn and antihero make a good point about Emperor mode

Actually, for anything that resembles production environment, I'd recommend using uWSGI in Emperor mode[1] to manage its instance than using Supervisor. Emperor mode will gracefully reload the app if config file's modification time changed (via `touch`) or stop and start the process automatically if you remove or add a configuration file.

My setup usually uses Upstart[2] (Ubuntu) or init script (Debian, etc.) to run uWSGI Emperor and make it scan /etc/uwsgi. When I need to deploy a new version, I just push the code then `touch /etc/uwsgi/myapp.ini` and my app is live with the code I just pushed.

[1] http://uwsgi-docs.readthedocs.org/en/latest/Emperor.html

[2] http://uwsgi-docs.readthedocs.org/en/latest/Upstart.html

Since most Python frameworks are just WSGI application (except when you go Tornado or Twisted route), you can easily apply deployment instruction for one framework with others. I've found Deploying Flask to be a good tutorial on how to deploy WSGI application:

http://flask.pocoo.org/docs/deploying/

There's also uWSGI, which is possibly the most underrated deployment option in Python world but Nginx has native support for it (and its documentation is pretty good nowadays):

http://uwsgi-docs.readthedocs.org/en/latest/WSGIquickstart.h...

Edit: all -> most

Not all Python web frameworks use WSGI. (e.g. Tornado).
Well, true. Let's change that to "most". :-)
It's the docs are poor, the actual procedure is very easy.

Eg. if you installed php on apache, your process may be to apt-get or build from source a mod_php then get the relevant LoadModule line into your apache config.

For python you'd do the same, except you'd use mod_wsgi instead of php.

Part of the problem with python is there are so many ways to achieve the same thing. No doubt others are reading this and thinking "no, you should do x" - hell i'm feeling that myself as i type! This highlights the fact there just is:

    1) Too little documentation
    2) Too little info on what approaches work best in what situations
I haven't yet deployed a big app with flask yet, but for my "playground" (Intranet App, very low load) I use Gunicorn (http://gunicorn.org/ ), which is extremely easy to set-up. For a deployment you would want to put nginx in front of it to serve static data.

If you just want do develop and test some things the server it comes with is absolutely enough and helps a lot with the debugging. Its as easy as typing "python myapp.py".

For deployment, it's also important to have something like nginx in front of Gunicorn to buffer requests from slow clients. Otherwise, this sort of thing can DoS your server pretty bad:

http://ha.ckers.org/slowloris/

I'll check out Gunicorn, thanks for the link.
Skip it and go with uWSGI ;) Emperor mode is fantastic.

Say you have /srv/mysite/uwsgi.ini and /srv/myothersite/uwsgi.ini (uwsgi config files for two different sites) you can run uwsgi --emperor /srv/*/uwsgi.ini and bam, it'll launch processes for all your sites.

I found myself similarly vexed when I first tried to deploy a Django project. Turns out I was making the process far more complicated by trying to wrestle with mod_wsgi and Apache.

I found that nginx + FastCGI[1] was by far the easiest configuration to get up and running. Took about 5 minutes to install the relevant packages and hack out a working configuration.

[1]http://wiki.nginx.org/HttpFastcgiModule

When you used php, your application server(mod_php) and web server(apache) looked the same to you. With python wsgi applications(most of the web frameworks are wsgi compliant), you generally need a separate application server and a web server which talks with the application server. Application server can directly serve the application and static files, but it isn't advisable.

So, you run an nginx on port 80 https://gist.github.com/rahulkmr/5232161 and proxy_pass to your application server running on some other port(8080 in the example).

You have a lot of choices when it comes to application servers. What you use depends on your use case.

> but I don't know how to configure nginx on my VPS so I can have multiple python webapps (my simple apps, django apps, flask etc.).

Look up sites-available and sites-enabled for nginx.

Have you considered using Heroku or Webfaction?

Apache + Django isn't too hard: the official docs cover it comprehensively: https://docs.djangoproject.com/en/1.5/howto/deployment/wsgi/...

Yes, I considered Webfraction. The thing is, I want to learn how to configure everyting by myself, because (1) I like learning and (2) I thnk it will be valuable lesson and let me understand better, how server work.

I'm more interested in nginx config, but I'll definetely read docs from the link you provided.

I have nginx + gunicorn controlled by supervisord running and I must admit coming from LAMP it went not as smooth as I hoped.

For me the main problem was that all these parts have to be configured correctly or nothing much will happen at all. My advice is to first set up a static site in nginx, then try starting your project (using gunicorn) from the command line. Once your sure that both nginx and gunicorn work and know how to do it you can Google around for a supervisord + gunicorn tutorial for managing your apps.

> I have nginx + gunicorn controlled by supervisord running and I must admit coming from LAMP it went not as smooth as I hoped.

If you are so inclined, you can do it using Apache + mod_wsgi. That is very similar to how you do Apache + mod_php

mod_wsgi usage is not recommended.
Do you mean mod_wsgi or mod_python?
Sorry, got confused :) Was thinking mod_python.
According to whom?
I come from a PHP background too and in my opinion Python is far easier to use, whole stack. I wrote about it here http://blog.jmoz.co.uk/learning-python-the-pragmatic-way
All I can say is: keep at it. If you plan on using Django in the long term, you'll eventually get to the point where you know the steps and they don't seem that bad anymore.
a good thing would be to have ubuntu python ready servers as images , so you can just drop it on a vpn, config a few stuffs and get started.