Hacker News new | ask | show | jobs
by nonethewiser 1174 days ago
Install dependency.

Save in requirements.txt. Includes sub dependencies that are system specific.

Install from requirements.txt on different machine and get errors.

Uninstall dependency and save to requirements.txt.

Look at requirements.txt and see that sub decencies are still there.

What is the right way to avoid these issues on Python?

6 comments

I use pip-compile in the pip-tools package:

Keep your high-level dependencies in requirements.in.

Create a virtual environment using venv.

Run `pip-compile resolver=backtracking`. It autogenerates a requirements.txt file with all dependencies and sub-dependencies and their versions. This essentially acts as a lock file from other languages/frameworks.

Install from the autogenerated requirements.txt file in the venv virtual environment.

If a dependency changes, change in requirements.in, recompile the .txt file, and reinstall.

>Run `pip-compile resolver=backtracking`

This is new for me. I used `pip freeze > requirements.txt`

My solution is to:

* use a virtualenv, so that dependencies are installed per project and not globally

* only specify my top-level dependencies in a requirements.in file, and let pip-compile (a dependency, part of pip-tools) compile my requirements.txt

So far it has served me well and I've not encountered any error due to this method.

I cannot claim it is "the right way" though, just something that works for me.

Specify your actual dependencies only in setup.py/cfg and only ever write your requirements file from pip freeze.

Or junk all this and just use poetry, which manages both the abstract dependencies (pyproject.toml) and concrete ones (poetry.lock).

Poetry seems to solve the dependency spec part, both for libraries that need to ship specs for supported version ranges of dependencies and apps that ship lock files.

However: poetry still falls short in managing the python runtime, I am continuously having to divert time to help our data scientists untangle the messes poetry makes with virtualenvs and their local python setups.

Also, they broke backwards compat on the lock file format? So now devs running newer poetry versions break projects for devs on older versions because the lock files aren’t compatible?!

Yes, I was very upset about the lockfile format change when we hit that; seemed like a very shortsighted thing to have done, but I guess I don't know the actual motivations or what was being achieved with it.
Use pipreqs instead of pip freeze. It resolves the minimal set of dependencies for your package.
Going from Windows to Linux wasn't a problem for me
containers