[zeotroph]

Ah, more complex than I thought: "venvstacks allows you to package Python applications and all their dependencies into a portable, deterministic format, without needing to include copies of these large Python frameworks in every application archive.", and in "Caveats and Limitations" (please, all projects should have one): "does NOT support combining arbitrary virtual environments with each other".

Is there a helper to merge venv1 and venv2, or create venv2 which uses venv1 dependencies and on load both are merged?

[zahlman]

The hard part is figuring out what "merge" means for your use case. If there's a definite set of packages that should be in the environment that are all already at definite locations on the local drive, there are many possible approaches (copying, `.pth` files, hard links, and symlinks should also work) to stitching together the venv you want. But you can't just feed the individual package paths to Python, because `sys.path` entries are places that Python will look for the top-level package folders (and top-level module `.py` files - which explains why), not the paths to individual importable things.

More importantly, at runtime you can only have one version of a given package, because the imports are resolved at runtime. Pip won't put multiple versions of the same library into the same environment normally; you can possibly force it to (or more likely, explicitly do it yourself) but then everyone that wants that library will find whichever version gets `import`ed and cached first, which will generally be whichever one is found first on `sys.path` when the first `import` statement is reached at runtime. (Yes, the problem is the same if you only have one venv in the first place, in the sense that your dependency graph could be unsolvable. But naively merging venvs could mean not noticing the problem until, at runtime, something tries to import something else, gets an incompatible version, and fails.)

[d0mine]

To create venv2 that uses venv1, define pep-735 dependencies groups in your pyproject.toml Specifically, a group can include other groups

uv supports groups and can create venv with the desired group set https://docs.astral.sh/uv/concepts/dependencies/#development...

For example, there can be "dev" group that includes "test", "mkdocs", "nuitka" groups (nuitka wants to be run with venv it builds binary for, so to keep venv minimal, it is in a separate group)

[HappMacDonald]

My understanding is that the entire reason that venv exists is because python's library system is nothing but dependency spaghetti: whatever is needed by one project conflicts with whatever is needed by another so you have to give them bespoke library environments where those conflicts won't interfere with one another.

In that perspective "merging" them directly defeats the purpose. What is needed is a better library ecosystem.

[zahlman]

venvs are used to isolate groups of dependencies, but it's not just about conflicts. Many other languages expect you to do the same thing; people complain less because either the language statically resolves imports and can support multiple versions of a library in the same environment; and/or because the ecosystem has some conventions that allow the tooling to detect the "current environment" better, and/or because the standard installer isn't itself a library that defaults to appearing in every environment and installing specifically for "its own" environment; and/or (possibly the biggest one) they don't have to worry about building and installing complex multi-language projects where the one they're using is just providing a binding.

An important reason for using them is to test deployment: if your code works in a venv that only has specific things installed (not just some default "sandbox" where you install everything), then you can be sure that you didn't forget to list a dependency in the metadata. Plus you can test with ranges of versions of your dependencies and confirm which ones your library will work with.

They're also convenient for making neat, isolated installations for applications. Pipx wraps pip and venv to do this, and as I understand it there's similarly uvx for uv. This is largely about making sure you avoid "interference", but also about pinning specific versions of dependencies. It also lowers the bar somewhat for your users: they still need to have a basic idea of what Python is and know that they have a suitable version of Python installed, but you can tell them to install Pipx if they don't have it and run a single install command, instead of having to wrestle with both pip and venv and then also know how to access the venv's console scripts that might not have been put on PATH.