[objectified]

I think that for the majority of use cases when it comes to packaging and shipping Python programs, we really want to ship OS packages. We don't want users (that includes sysadmins/operations teams in this case) to deal with having pip available, understanding what virtualenv is, and so on. We just want them to be able to use apt, yum, or whatever they experience as most common on their system. Yet at the same time, we don't want our application to take in regard ancient versions of OS packages to depend on.

I think in the end we want a self contained OS package, to offer a sane install method for any target OS. Why would users have to make a difference between e.g. a C program that they install through apt, or a Python program that all of a sudden requires a completely different way of installing?

Here are some interesting thoughts on it: https://hynek.me/articles/python-app-deployment-with-native-...

And here's a tool that I wrote to be able to wrap up a virtualenv into an OS package, while specifying a precise list of OS dependencies. It's not perfect by far, and admittedly a bit crude, but it already does the job well for some real life "production" stuff. It's called vdist (virtualenv distribution).

https://github.com/objectified/vdist

Documentation is here:

https://vdist.readthedocs.org/en/latest/

[iofj]

Software engineers often have a need to have their package work on multiple distributions - or even OSes, which they may know little about. They maintain the program, and try really hard not to care what it runs on. They need the SSL bug fixed on both redhat, debian and mac os X. Hence the "static linking" solutions where all the packages are controlled from the program, like virtualenv/pip.

System engineers are charged with keeping various programs running on (usually) a fixed distribution/OS. They need to have a way to update SSL when there is a security issue, for all applications. They need to be able to update various libraries and have programs transparently use new or otherwise fixed versions of libraries. They also very much need the ability to uninstall packages.

OS level packages are a solution written to serve the need of sysadmins. But software engineers don't care, because maintaining the correct package list and versions for 4-5 distributions is horrible. So they don't.

What I don't get is why docker isn't the ideal solution for both groups. Assuming, of course, sysadmins have both ability and the inclination to insist on the sources.

[curryst]

> What I don't get is why docker isn't the ideal solution for both groups.

For some things, Docker is a great solution for both groups, but that requires that your organization and application meet a list of requirements. Is the application only going to be run by you, as a server? If it's going to be run elsewhere, you may not have the option to require it run on Docker or if it's going to be run as a client then you definitely don't have the option.

The conflicting goals come from the System Engineers goal of making it run on all platforms vs the System Administrators goal of making it run on this platform. System Engineers want a solution that makes it easier to abstract away the platform, System Administrators want a solution to make it easier to install on this platform. OS packages are easier as a System Administrator because it all ties into a central source of authority. Each package knows which other packages are installed on the system and which packages it needs. Fragmenting the environment makes it more difficult because you've got distinct sources of authority. Pip package A needs version 1.2 of OS package B, but pip package C needed for pip package D needs version 1.3 of OS package B. All of which you'd have no idea about until it all crashes and burns and you have to go figure out why.

OS packages are usually better in terms of installation, but pip packages are easier to create. Ubuntu's new packaging system is meant to address this, though we'll see how it goes.