[0cf8612b2e1e]

I believe the charitable interpretation is that it is not possible without breaking an enormous amount of legacy code. Which does feel close enough to “not possible”.

Some situations could be improved by allowing multiple library versions, but this would introduce new headaches elsewhere. I certainly do not want my program to have N copies of numpy, PyTorch, etc because some intermediate library claims to have just-so dependency tree.

[driggs]

What do you do today to resolve a dependency conflict when an intermediate library has a just-so dependency tree?

The charitable interpretation of this proposed feature is that it would handle this case exactly as well as the current situation, if the situation isn't improved by the feature.

This feature says nothing about the automatic installation of libraries.

This feature is absolutely not about supporting multiple simultaneous versions of a library at runtime.

In the situation you describe, there would have to be a dependency resolution, just like there is when installing the deps for a program today. It would be good enough for me if "first import wins".

[zahlman]

> What do you do today to resolve a dependency conflict when an intermediate library has a just-so dependency tree?

When an installer resolves dependency conflicts, the project code isn't running. The installer is free to discover new constraints on the fly, and to backtrack. It is in effect all being done "statically", in the sense of being ahead of the time that any other system cares about it being complete and correct.

Python `import` statements on the other hand execute during the program's runtime, at arbitrary separation, with other code intervening.

> This feature says nothing about the automatic installation of libraries.

It doesn't have to. The runtime problems still occur.

I guess I'll have to reproduce the basic problem description from memory again. If you have modules A and B in your project that require conflicting versions of C, you need a way to load both at runtime. But the standard import mechanism already hard-codes the assumptions that i) imports are cached in a key-value store; ii) the values are singleton and client code absolutely may rely on this for correctness; iii) "C" is enough information for lookup. And the ecosystem is further built around the assumption that iv) this system is documented and stable and can be interacted with in many clever ways for metaprogramming. Changing any of this would be incredibly disruptive.

> This feature is absolutely not about supporting multiple simultaneous versions of a library at runtime.

You say that, but you aren't the one who proposed it. And https://news.ycombinator.com/item?id=45467350 says explicitly:

> and support having multiple simultaneous versions of any Python library installed.

Which would really be the only reason for the feature. For the cases where a single version of the third-party code satisfies the entire codebase, the existing packaging mechanisms all work fine. (Plus they properly distinguish between import names and distribution names.)

[driggs]

> and support having multiple simultaneous versions of any Python library installed.

Installed. Not loaded.

The reason is to do away with virtual environments.

I just want to say `import numpy@2.3.x as np` in my code. If 2.3.2 is installed, it gets loaded as the singleton runtime library. If it's not installed, load the closest numpy available and print a warning to stderr. If a transient dependency in the runtime tree wants an incompatible numpy, tough luck, the best you get is a warning message on stderr.

You already have the A, B, C dependency resolution problem you describe today. And if it's not caught at the time of installing your dependencies, you see the failure at runtime.

[zahlman]

You'd have to invent a different way, within existing Python syntax, to communicate the version, but you can do this today with sys.path and sys.meta_path hacks.

But virtual environments are quite simply not a big deal. Installed libraries can be hard-linked and maybe even symlinked between environments and this can be set up very quickly. A virtual environment is defined by the pyvenv.cfg marker file; you don't need to use or even have activation scripts, and you especially don't (generally) need a separate copy of pip for each one, even if you do use pip.

On the flip side, allowing multiple versions of a library in a virtual environment has very little effect on package resolution; it just allows success in cases of conflict, but normally there aren't conflicts (because you're typically making a separate environment for a single "root" package, and it's supposed to be possible to use that package in Python as it actually exists, without hacks). The installer still has to scrounge up metadata (and discover it recursively) and check constraints.