Ask HN: Venv, virtualwrapper, pipenv, poetry, pdm, oh my

[p33p]

Dependency and package management has seen a lot growth and attention in the Python community even in the last couple years. Everyone has their opinions on what to use, but for enterprise code bases, what is the recommended approach?

Ideally responses would include size of user base, number of package dependencies, any issues arisen, etc.

Thanks y’all. Needless to say, the most recent pipenv post on the front page has led to more confusion than I’d like to admit.

[akagusu]

Unfortunately, there is no official way to handle dependencies in Python projects.

You should use what is best for you and the only way to find it is to try all of them.

If you find this a mess, it is because it is.

Python dependency management is a mess and it seems it will not get better any time soon, unfortunately.

This is because none of them are good enough.

There is a trend towards Poetry right now, and people seem to like it, but there are also people that complain it is not the right tool for some kind of projects. It deserves a try.

I tested PDM but it follows an different approach that Python community is used to and because of this it does not get the same attention as other tools.

Pipenv was a tool with lots of promises that got aggressively marketed as the "official" way to manage Python dependencies but it failed to delivery its promises and it seems the community moved from it.

[speedgoose]

I ignore these and go straight to software containers (docker, podman, kubernetes,…) because my hammer works well.

[emptysongglass]

So just using pip inside the container? Microsoft is looking to standardize development containers [1], a cleanroom environment where you live while you develop. That container can itself be used as a workbench for then packaging your application into a container of its own for deployment on K8s or what have you.

[1] https://code.visualstudio.com/docs/remote/containers

[speedgoose]

Yes, normal standard pip with no risks to break anything.

[roflchoppa]

We use `pyenv` in order to manage python installations (it helps when you want to test different versions manually installing and managing can be done too but is a pain)

We force the use of `virtualenv` so that within our project we can nuke and rebuild the installed dependencies quickly.

There are multiple requirements.txt files, the first one that used for installation installs the pined version of `pip`, and `setuptools`.

After that additional dependencies are installed (separate file).

One of the senior engineers I used to work with wanted me to checkout `pip compile`. It looks useful.

[bravetraveler]

It's a mess in general

The best balance I've managed is with Linux - distributions bundle up most everything I need as packages, and I'm lucky there's very little requirement for specific versions for what I need

Anything extra I tend to get with pip/--user

[PrivateButts]

We've been using pipenv + pyenv at work for years, and it's fine to be honest. On Linux, I don't really have a problem with either tool anymore, on Windows it's still a bit buggy. Most of it comes from the pipenv pyenv integration not working automatically. I'd try a couple and find what works best for your workflow.

Not saying python package management couldn't be better, but I think that a lot of pipenv's hate comes from earlier versions, and is a bit overblown.

[p33p]

I should have mentioned that we have developers on Windows, MacOS, and Linux -- which is possibly where my frustration comes from with the struggles of integrating Windows as a development environment.

[cpressland]

On macOS it’s pretty good, just requires a few extra packages and compiler flags for pyenv to work well.

I’ve found pipx to be the most reliable way to manage the installation of pipenv.

[laerus]

For development, poetry is the most user friendly tool and it's also battle tested. For application deployment/installation pipx "just works" and it's an official PyPA project inspired by npx.

I don't know why people are saying it's not going to get better but there is an ongoing effort to actually become better but as with everything it goes through the PEP process which is a slow.

[Kaze404]

I don’t personally write Python often, especially not in an enterprise context, but from my (admittedly narrow) understanding all these tools try to do what Nix already does. You could look into that.

[emptysongglass]

You can use Poetry and Nix together with Poetry2nix! [1]

[1] https://www.tweag.io/blog/2020-08-12-poetry2nix/

[Kaze404]

Whoa, now that’s cool. The extent of my experience with Python and Nix is using a shebang line with nix-shell to specify the dependencies of my script, but I can see this being very useful for actual projects. Thank you!

[aristofun]

Python devx has always been a mess. Unfortunately.

[mejutoco]

pipenv install, pipenv lock and docker installing the lock file with pip.

Never had to think about which virtual environment I am using again.