[Waterluvian]

I ran into a migraine last week: cleaning up requirements.txt

How do you determine which requirements are no longer needed when you remove one from your code? In node, your package.json lists only packages YOU installed. So removing them cleans up their dependencies. But in Python, adding one package with pip install might add a dozen entries, none indicating they're dependencies of other packages.

[tleguijt]

At most projects we're using pip-tools which generates a fully pinned requirements.txt based on a manually kept (and clean) requirements.in which only contains the specific packages you need without their dependencies

[Waterluvian]

Thanks. I'll investigate this method. It sounds like you hand write dependencies and their versions into the requirements.in file?

[sciurus]

The idea is described at https://www.kennethreitz.org/essays/a-better-pip-workflow

[ntnn]

Use requirements.txt volatile.

We use a separate file to list the direct dependencies, 'ddeps.txt' and 'ddeps-dev.txt' for development deps.

Once we update one of these files a clean venv is created, the dependencies installed and the freeze output saved as requirements.txt. Then the dev dependencies are installed and the output of that freeze is saved to requirements-dev.txt.

This preserves the dependencies where we made the conscious choice to require them and also allows us to explicitly vet any new dependencies and versions.

[neuland]

I’m not sure about other people, but that is how I use requirements.txt. You don’t have to dump the entire output of pip freeze in there. You can just list the dependencies you want.

[striking]

Or you can list direct dependencies in another file and regenerate requirements.txt with `pip freeze` whenever you change the other file. Especially easy with Make.

[web007]

It really bothers me that they're skipping these two as separate steps. Track "what I asked for", use "what I ended up with" for deployment. Otherwise you're just saying "use pip freeze" regardless of wrapping magic around it.

If you're already down that road, pipdeptree is your friend. It will resolve your frozen packages to at least tell you which are top-level and which are dependencies-of-dependencies. There are still exceptions if you're using a dependency both directly and via another module, but having a requirements.in from the pipdeptree parents will have you covered.

Get that list, set them all to module>=version in development, pip install -r requirements.in, then pip freeze > requirements.txt to get hard version locks for deployment.

As others have stated, pip-tools handles this separation for you.