[benrutter]

I thought that too, but I think the tricky bit is if you're a non-python user, this isn't yet obvious.

If you've never used Clojure and start a Clojure project, you will almost definitely find advice telling you to use Leiningen.

For Python, if you search online you might find someone saying to use uv, but also potentially venv, poetry or hatch. I definitely think uv is taking over, but its not yet ubiquitous.

Ironically, I actually had a similar thing installing Go the other day. I'd never used Go before, and installed it using apt only to find that version was too old and I'd done it wrong.

Although in that case, it was a much quicker resolution than I think anyone fighting with virtual environments would have.

[idoubtit]

That's my experience. I'm not a Python developer, and installing Python programs has been a mess for decades, so I'd rather stay away from the language than try another new tool.

Over the years, I've used setup.py, pip, pipenv (which kept crashing though it was an official recommendation), manual venv+pip (or virtualenv? I vaguely remember there were 2 similar tools and none was part of a minimal Python install). Does uv work in all of these cases? The uv doc pointed out by the GP is vague about legacy projects, though I've just skimmed through the long page.

IIRC, Python tools didn't share their data across projects, so they could build the same heavy dependencies multiple times. I've also seen projects with incomplete dependencies (installed through Conda, IIRC) which were a major pain to get working. For many years, the only simple and sane way to run some Python code was in a Docker image, which has its own drawbacks.

[lexicality]

> Does uv work in all of these cases?

Yes. The goal of uv is to defuck the python ecosystem and they're doing a very good job at it so far.

[prox]

What are the big offenders right now? What does uv unfuck?

I only work a little bit with python.

[lexicality]

In my experience every other python tool has a variety of slightly to extremely painful behaviours that you have to work around or at least be aware of.

Sometimes it's things like updating to Fedora 43 and every tool you installed with `pipx` breaking because it was doing things that got wiped out by the system upgrade, sometimes it's `poetry update --only dep1` silently updating dep2 in the background without telling you because there was an update available and even though you specified `--only` you were wrong to do that and Poetry knows best.

Did you know that when you call `python -m venv` you should always pass `--upgrade-deps` because otherwise it intentionally installs an out of date version of pip and setuptools as a joke? Maybe you're not using `python -m venv` because you ran the pyenv installer and it automatically installed `pyenv-virtualenv` without asking which overrides a bunch of virtualenv features because the pyenv team think you should develop things in the same way they do regardless of how you want to delevop things. I hate pyenv.

So far the only problem I've had with uv is that if you run `uv venv` it doesn't install pip in the created virtualenv because you're supposed to run `uv pip install` instead of `pip install`. That's annoying but it's not a dealbreaker.

Outside of that, I feel very confident that I could give a link to the uv docs to a junior developer and tell them to run `uv python install 3.13` and `uv tool install ruff` and then run `uv sync` in a project and everything will work out and I'm not going to have to help them recover their hard drive because they made the foolish mistake of assuming that `brew install python` wouldn't wreck their macbook when the next version of Python gets released.

[lucideer]

uv not only completely replaces all of pip, pyenv & venv, but it also does a much better job than any of them at their intended function, as well as a bunch of other convenient, simple developer-friendly features.

1. pip isn't entirely to blame for all of Python's bad package management - distutils & setuptools gave us setup.py shenanigans - but either way, UV does away with that in favour of a modern, consistent, declarative, parseable PEP 508 manifest spec, along with their own well-designed lockfile (there was no accepted lockfile PEP at the time UV was created - since PEP 715 has become accepted UV has added support, though that PEP is still limited so there's more work to do here).

2. pyenv works fine but uv is faster & adds some nice extra features with uvx

3. venv has always been a pain - ensuring you're always in the right venv, shell support, etc. uv handles this invisibly & automatically - because it's one tool you don't need to worry about running pip in the right venv or whatever.

[kevin_thibedeau]

pip and venv. The Python ecosystem has taken a huge step backwards with the preachy attitude that you have to do everything in a venv. Not when I want to have installable utility scripts usable from all my shells at any time or location.

I get that installing to the site-packages is a security vulnerability. Installing to my home directory is not, so why can't that be the happy path by default? Debian used to make this easy with the dist-packages split leaving site-packages as a safe sandbox but they caved.

[kstrauser]

Regarding why not your home directory: which version of Foo do you install, the one that Project A needs or the incompatible one that Project B needs?

The brilliant part about venvs is that A and B can have their completely separate mutually incompatible environments.

[kevin_thibedeau]

They have their place. But the default shouldn't force you into a "project" when you want general purpose applicability. Python should work from the shell as readily as it did 20 years ago. Not mysteriously break what used to work with no low-friction replacement.

[andoando]

Why can't we just have something like npm/gradle/maven dependencies? What makes python any different?

[halostatue]

For years, pipx did almost all the work that I needed it to do for safely running utility scripts.

uv has replaced that for me, and has replaced most other tools that I used with the (tiny amount of) Python that I write for production.

[pxc]

> Not when I want to have installable utility scripts usable from all my shells at any time or location.

Can't you just have the thing on your PATH be a wrapper that invokes the tool via its venv?

[wtallis]

That's what `uv tool install` does: it creates the wrapper and puts a symlink to it into ~/.local/bin (which you can add to PATH with `uv tool update-shell` if you don't want to do it manually). I don't recall pip doing anything helpful here; I think it still leaves it up to the end user to either add the venv's bin directory to their PATH or create the wrapper and put it somewhere already on the PATH. So it's a reasonable complaint that `pip install` has become less useful now that it resists installing tools outside of a venv but still lacks the replacement feature (which third party tools like uv and pipx do provide).

[hexo]

It unfucks nothing because it wasn't fuckd in the first place. Whole uv is solution to non existing problem.

[aeurielesn]

That's giving way too much credit to uv.

[kraddypatties]

I'm interpreting this as "uv was built off of years of PEPs", which is true; that being said the UX of `uv` is their own, and to me has significantly reduced the amount of time I spend thinking about requirements, modules, etc.

[karel-3d]

uv is really that good.

[hexo]

If so, ok, let's port this prototype to back to python and get rid of uv.

[llbeansandrice]

What does this comment mean? Port the dependency and virtual environment manager back to the language?

Should we port npm “back” to node js?

[lexicality]

They've definitely not done it yet, but they're getting there.

[NeutralCrane]

It really isnt

[simonw]

> IIRC, Python tools didn't share their data across projects, so they could build the same heavy dependencies multiple times.

One of the neatest features of uv is that it uses clever symlinking tricks so if you have a dozen different Python environments all with the same dependency there's only one copy of that dependency on disk.

[zahlman]

Hard links, in fact. It's not hard to do, just (the Rust equivalent of) `os.link` in place of `os.copy` pretty much. The actually clever part is that the package cache actually contains files that can be used this way, instead of just having wheels and unpacking them from scratch each time.

For pip to do this, first it would have to organize its cache in a sensible manner, such that it could work as an actual download cache. Currently it is an HTTP cache (except for locally-built wheels), where it uses a vendored third-party library to simulate the connection to files.pythonhosted.org (in the common PyPI case). But it still needs to connect to pypi.org to figure out the URI that the third-party library will simulate accessing.

[runjake]

I would not be putting up with Python if not for uv. It’s that good.

Before uv came along I was starting to write stuff in Go that I’d normally write in Python.

[QuercusMax]

Coming from a mostly Java guy (since around 2001), I've been away from Python for a while and my two most recent work projects have been in Python and both switched to uv around the time I joined. Such a huge difference in time and pain - I'm with you here.

Python's always been a pretty nice language to work in, and uv makes it one of the most pleasant to deal with.

[runjake]

I don't even like Python as a language (it's growing on me, but only a little).

It's just so useful: uv is great and there are decent quality packages for everything imaginable.

[kristianbrigman]

That's partly because python has a very large installed base, and ease of entry (including distribution). This leads to people running into issues quicker, and many alternative solutions.

Unlike something like Rust, which has much fewer users (though growing) and requires PhDs in Compiler Imprecation and Lexical Exegetics.

Or C++ which has a much larger installed base but also no standard distribution method at all, and an honorary degree in Dorsal Artillery.

[whimsicalism]

uv solved it, it’s safe to come back now.

[regularfry]

There's definitely a philosophical shift that you can observe happening over the last 12-15 years or so, where at the start you have the interpreter as the centre of the world and at the end there's an ecosystem management tool that you use to give yourself an interpreter (and virtual environments, and so on) per project.

I think this properly kicked off with RVM, which needed to come into existence because you had this situation where the Ruby interpreter was going through incompatible changes, the versions on popular distributions were lagging, and Rails, the main reason people were turning to Ruby, was relatively militant about which interpreter versions it would support. Also, building the interpreter such that it would successfully run Rails wasn't trivial. Not that hard, but enough that a convenience wrapper mattered. So you had a whole generation of web devs coming up in an environment where the core language wasn't the first touchpoint, and there wasn't an assumption that you could (or should) rely on what you could apt-get install on the base OS.

This is broadly an extremely good thing.

But the critical thing that RVM did was that it broke the circular dependency at the core of the problem: it didn't itself depend on having a working ruby interpreter. Prior to that you could observe a sort of sniffiness about tools for a language which weren't implemented in that language, but RVM solved enough of the pain that it barged straight past that.

Then you had similar tools popping up in other languages - nvm and leiningen are the first that spring to mind, but I'd also throw (for instance) asdf into the mix here - where the executable that you call to set up your environment has a '#!/bin/bash' shebang line.

Go has sidestepped most of this because of three things: 1) rigorous backwards compatibility; 2) the simplest possible installation onramp; 3) being timed with the above timeline so that having a pre-existing `go` binary provided by your OS is unlikely unless you install it yourself. And none of those are true of Python. The backwards compatibility breaks in this period are legendary, you almost always do have a pre-existing Python to confuse things, and installing a new python without breaking that pre-existing Python, which your OS itself depends on, is a risk. Add to that the sniffiness I mentioned (which you can still see today on `uv` threads) and you've got a situation where Python is catching up to what other languages managed a decade ago.

Again.

[bee_rider]

It is sort of funny, if we squint just the wrong way, “ecosystem management tool first, then think about interpreters” starts to look a lot like… a package manager, haha.

[zahlman]

> you might find someone saying to use uv, but also potentially venv, poetry or hatch.

This is sort of like saying "You might find someone saying to drive a Ford, but also potentially internal combustion engine, Nissan or Hyundai".

[evilduck]

Only to those already steeped in Python. To an outsider they're all equally arbitrary non-descriptive words and there's not even obvious proper noun capitalization to tell apart a component from a tool brand.

[zahlman]

It's always rather irritating to me that people make these complaints without trying to understand any of the under-the-hood stuff, because the ultimate conclusion is that it's somehow a bad thing that, on a FOSS project, multiple people tried to solve a problem concurrently.

[NetMageSCW]

That’s especially ironic given that inside Python part of the philosophy is “There should be one-- and preferably only one --obvious way to do it.” So why does Python’s external environment seem more like something that escape from a Perl zoo?

[hexo]

Because a lot of people have no clue about packaging or how to write compatible software, one that is actually installable as normal application. I suspect a lot of them learned stuff in node.js or ruby ecosystem first and this is the result. Same as requiring using docker to install or build an application. It isn't cool, funny or right way to do stuff. I still don't get what was so wrong about venv that anyone needed uv. I have no need to even try and i'm writing python stuff so long that i cannot even estimate it. To me it feels like reinvention for sake of rewrite in rust. If it is so good, ok, i get it, it might be - and all that good stuff needs to go back to python as python.

[zahlman]

> I still don't get what was so wrong about venv that anyone needed uv.

Pip is slow, far slower than it needs to be in almost everything that it does, regardless of being written in Python. It's "standard" but not part of the standard library (so that it can be developed independently), and was never designed to install cross-environment properly (the current best approach, since 22.3, is a hack that incurs a significant delay and expects everyone to move in lock-step with the CPython EOL schedule). It wastes disk space, both by re-copying packages into new environments (rather than hard-linking them as uv does) and by spawning copies of itself in those environments (the original work-around to avoid needing cross-environment installation support, which a few people have also come to rely on in other ways).

> If it is so good, ok, i get it, it might be - and all that good stuff needs to go back to python as python.

I like these threads because they encourage me to work on my main project.

[kstrauser]

The one obvious way is the underlying virtualenv abstraction. Everything else just makes that part easier or more transparent.

[zahlman]

What kstrauser said.

But with much more detail: it seems complicated because

* People refuse to learn basic concepts that are readily explained by many sources; e.g. https://chriswarrick.com/blog/2018/09/04/python-virtual-envi... [0].

* People cling to memories of long-obsolete issues. When people point to XKCD 1987 they overlook that Python 2.x has been EOL for almost six years (and 3.6 for over four, but whatever)[1]; only Mac users have to worry about "homebrew" (which I understand was directly interfering with stuff back in the day) or "framework builds" of Python; easy_install is similarly a long-deprecated dinosaur that you also would never need once you have pip set up; and fewer and fewer people actually need Anaconda for anything[2][3].

* There is never just one way to do it, depending on your understanding of "do". Everyone will always imagine that the underlying functionality can be wrapped in a more user-friendly way, and they will have multiple incompatible ideas about what is the most user-friendly.

But there is one obvious "way to do it", which is to set up the virtual environment and then launch the virtual environment's Python executable. Literally everything else is window dressing on top of that. The only thing that "activating" the environment does is configure environment variables so that `python` means the virtual environment's Python executable. All your various alternative tools are just presenting different ways to ensure that you run the correct Python (under the assumption that you don't want to remember a path to it, I guess) and to bundle up the virtual environment creation with some other development task.

The Python community did explicitly provide for multiple people to provide such wrappers. This was not by providing the "15th competing standard". It was by providing the standard (really a set of standards designed to work together: the virtual environment support in the standard library, the PEPs describing `pyproject.toml`, and so on), which replaced a Wild West (where Setuptools was the sheriff and pip its deputy).

[0]: By the way, this is by someone who doesn't like virtual environments and was one of the biggest backers of PEP 582.

[1]: Of course, this is not Randall Munroe's fault. The comic dates to 2018, right in the middle of the period where the community was trying to sort things out and figure out how to not require the often problematic `setup.py` configuration for every project including pure-Python ones.

[2]: The SciPy stack has been installable from wheels for almost everyone for quite some time and they were even able to get 3.12 wheels out promptly despite being hamstrung by the standard library `distutils` removal.

[3]: Those who do need it, meanwhile, can generally live within that environment entirely.

[fwip]

I imagine by this they meant `python -m venv` specifically, using that interface directly, rather than through another wrapper CLI tool.

[benrutter]

Yes, that's exactly what I meant! Sorry if it wasn't clear. In my experience this used to be easily the most popular method up until a couple years ago.

[zahlman]

Fair.

The way I teach, I would start there; then you always have it as a fallback, and understand the system better.

I generally sort users into aspirants who really should learn those things (and will benefit from it), vs. complete end users who just want the code to run (for whom the developer should be expected to provide, if they expect to gain such a following).

[houzi]

Do you think a non-python user would piece it together if the shebang line reveals what tool to use?

[benrutter]

I think yes if that line was UV. But otherwise, of its just python, you have the issue that you need two tools, one for running scripts and one for managing dependencies and environments.

[MarsIronPI]

> If you've never used Clojure and start a Clojure project, you will almost definitely find advice telling you to use Leiningen.

I thought the current best practice for Clojure was to use the shiny new built-in tooling? deps.edn or something like that?

[fulafel]

Clojure CLI (aka deps.edn) came out in 2018 and in the survey "how do you manage your dependencies?" question crossed 50% usage in early 2020. So for 6-8 years now.

[benrutter]

Ah woops! Sorry for my outdated clojure knowledge, this sounds cool though- I'll give it a go!

[codemonkey-zeta]

deps.edn is becoming the default choice, yes. I interpreted the parent comment as saying "you will see advice to use leiningen (even though newer solutions exist, simply because it _was_ the default choice when the articles were written)"

[NeutralCrane]

uv has been around for less than two years. It’s on track to become the default choice, it’s just a matter of time.