Hacker News new | ask | show | jobs
by zys5945 2505 days ago
I think he is referring to indirect dependencies
2 comments

Yes, those are caught when using pip freeze.
> Yes, those are caught when using pip freeze.

No they are not.

Pip freeze does not resolve transitive dependencies, nor does pip know what to do if your transitive dependencies conflict with each another.

> Pip freeze does not resolve transitive dependencies

I don't think this is correct:

    $ python3 -m venv /tmp/v
    $ /tmp/v/bin/pip install flask
    [...]
    Collecting MarkupSafe>=0.23 (from Jinja2>=2.10.1->flask)
    [...]
    $ /tmp/v/bin/pip freeze | grep MarkupSafe
    MarkupSafe==1.1.1
> nor does pip know what to do if your transitive dependencies conflict with each another

This is true, but because Python exposes all libraries in a single namespace at runtime, there isn't actually anything reasonable to do if they genuinely conflict. You can't have both, say, MarkupSafe 1.1.1 and MarkupSafe 1.1.0 in PYTHONPATH and expect them to be both accessible. There's no way in an import statement to say which one you want.

However, it's notable that pip runs into trouble in cases where transitive dependencies don't genuinely conflict, too. See https://github.com/pypa/pip/issues/988 - this is a bug / acknowledged deficiency, and there is work in progress towards fixing it.

> There's no way in an import statement to say which one you want.

This would be fixable with a sys path hook, were pip so inclined

It would change the semantics of the language. You could also write a sys.path hook to interpret the remainder of the file as Ruby and not Python, were pip so inclined....

(Also it's not clear what those changed semantics would be.)

Import system is pluggable, so the semantics are there to be customized. Sure, it could be abused (as many things in Python), but an import hook that checks for a vendored dependency with specific version, seems like a reasonable way to resolve the problem above.

> remainder of the file as Ruby and not Python

That's a little excessive

>Pip freeze does not resolve transitive dependencies

How? Doesn't pip freeze literally list all packages that's installed in the current environment besides basic toolings such as setuptools (and you could even instruct it to list those as well)?

I'm not sure about conflict resolution when but I run a pip freeze it adds TONS of dependencies outside of the 2-3 I had in my app because those were the dependencies of my dependencies.
I think that is what you want. Having all your dependencies, including their dependencies, explicitly specified (including name and version) is what gives you reproducible builds.

Ruby does the same thing with Gemfile.lock. npm does the same thing with package-lock.json.

They are, but pip doesn't know if they are transitive dependencies or not or if they conflicted at install time. Pip-tools helps with this.
>>> Why isn't there any builtin way to automatically define a lock file

pip isn't actually part of Python proper.

Nowadays, a regular installation of CPython lets you run "python -m pip". It's quite part of it.
You can do that with any library. You can issue Django commands by running `python -m django`; that doesn't change the fact that Django is a completely separate project from Python.
It isn't part of the Python executable, but it is part of the standard distribution.