[acemarke]

For whatever reason, I've never understood the point or actual benefit of virtualenvs.

Having lived more in the JS ecosystem for the last several years, my ideal workflow would be a copy of how the Yarn package manager works:

- Top-level dependencies defined in an editable metadata file

- Transitive dependencies with hashes generated based on the calculated dependency tree

- All dependencies installed locally to the project, in the equivalent of a `node_modules` folder

- All package tarballs / wheels / etc cached locally and committed in an "offline mirror" folder for easy and consistent installation

- Attempting to reinstall packages when they already are installed in that folder should be an almost instantaneous no-op

PEP-582 (adding standard handling for a "__pypackages__" folder) appears to be the equivalent of `node_modules` that I've wanted, but tools don't seem to support it yet. I'd looked through several Python packaging tools over the last year, but none of them seemed to support it yet (including Poetry [0]).

The only tool that I can find that really supports PEP-582 atm is `pythonloc` [1], which is really just a wrapper around `python` and pip` that adds that folder to the path. Using that and `pip-tools`, I was able to mostly cobble together a workflow that mimics the one I want. I wrote a `requirements.in` file with my main deps, generated a `requirements.txt` with the pinned versions and hashes with `pip-compile`, was able to download and cache them using `pip`, and installed them locally with `piploc`.

Admittedly, I've only tried this out once a few weeks ago on an experimental task, but it seemed to work out sufficiently, and I intend to implement that workflow on several of our Python services in the near future.

If anyone's got suggestions on better / alternate approaches, I'd be interested.

[0] https://github.com/python-poetry/poetry/issues/872

[1] https://github.com/cs01/pythonloc

[2] https://github.com/jazzband/pip-tools

[Izkata]

> - All dependencies installed locally to the project, in the equivalent of a `node_modules` folder

That's basically what a virtualenv is, the python version of a node_modules directory. You just need to explicitly activate/deactivate to move in and out of that environment, instead of it being based on the directory tree. And the requirements file is equivalent to the "dependencies" section(s) of package.json

[dragonsh]

Node and npm are no better than JavaScript tools used for dependency management. I had a project in node 6.0 refuse to work with node 8 and npm failed to upgrade. No different than python 2.7 to Python 3.

Just browser and JavaScript silence the errors and might result in things cannot be seen. This is not the case with Python “No errors must pass silently unless explicitly silenced”.

[uranusjr]

Totally agree; node_modules starts to break horribly when you use the wrong node/npm command to access it. This is unfortunately causing a lot of misconceptions because people usually judge tools by short-term experiementations. node_modules works very well in that scenario and gets the most love--until it doesn’t, you tear hair for those nasty bugs, and sadly can’t even convince people with your experience (“it works perfectly for me, I don’t know what you’re talking about.”)

I guess the node_modules approach is fine for Node, since the native binding story is quite bleak and no very people are deep into it anyway. But for Python this would be a complete no-go given how pervasive extension modules are. I don’t know what went through Ian Bicking’s mind when he first designed virtualenv, but so many aspects of it just reflects that tight cross-language integration Python is so good at (the bin/include/lib structure, copying/symlinking binaries into place, setting up env vars to trick foreign build systems into working, etc.). I wouldn’t want it any other way even if I could go back in time and involve in its original design. Well, maybe a few minor improvements, like offer a more beginner-frieldly interface than activate scripts, or more heavily promote running the binaries directly, but the design is just… correct.