[Alir3z4]

Almost 2 decades of working with python.

I create a venv. Pip install and keep my direct deps in requirements.txt

That's it. Never understood all these python dependency management problems dramas.

Recently, I started using pyproject.toml as well which makes the whole thing more compact.

I make lots of python packages too. Either I go setup.py or sometimes I like to use flit for no specific reason.

I haven't ever felt the need for something like uv. I'm good with pip.

[Borealid]

That doesn't work well (enough) if you have one project that requires Python <3.10 and another that requires Python >=3.10.

To really pin everything you'd need to use something like asdf, on top of poetry or a manual virtualenv.

Otherwise you get your colleagues complaining that pip install failed with mysterious errors.

[nyrikki]

venvs are namespace isolation, they are like containers.

Even in huge monorepos you can just use something like a Makefile to produce a local venv using PHONY and add it to clean too

This is how I actually test old versions of python, with versioned build targets, cython vs ...

You can set up almost any IDE to activate them automatically too.

The way to get you coworkers to quit complaining is to automate the build env setup, not fighting dependency hell, which is a battle you will never win.

It really is one of the most expensive types of coupling.

[alkh]

Would recommend you to install pyenv[1]. It was very useful when my team had to update a lot of projects using <=3.10 to 3.11 [1] https://github.com/pyenv/pyenv

[zahlman]

I have multiple versions of Python built from source. If I want to test what my code will do on a given version, I spin up a new venv (near instantaneous using `--without-pip`, which I achieve via a small Bash wrapper) and try installing it (using the `--python` option to Pip, through another wrapper, allowing me to reuse a single global copy).

No matter what tooling you have, that kind of test is really the only way to be sure anyway.

If something doesn't work, I can play around with dependency versions and/or do the appropriate research to figure out what's required for a given Python version, then give the necessary hints in my `pyproject.toml` (https://packaging.python.org/en/latest/specifications/pyproj...) as environment markers on my dependency strings (https://peps.python.org/pep-0508/#environment-markers).

"Mysterious errors" in this area are usually only mysterious to end users.

[marky1991]

I don't get it, you ought to be building a different venv per project anyway.

(Of course, I don't distribute most of my projects, so I just dump them all in the global install and don't worry about it)

[redserk]

Some libraries break across different versions of Python for a variety of reasons.

Pinning python version with asdf (in conjunction with a venv) gets you reasonably far in ensuring a certain project works across a lot of people in a team.

[nyrikki]

Venvs solve that problem, they don't cause it.

You activate the venv per project to get out of the global namespace problem.

As python x.nn versions are major with the nn part, it is like trying to mix c11 with c19.... things will break if you don't respect that.

[Borealid]

A venv does not actually install a different Python interpreter. It's bound to the Python version that created it. You cannot make a Python 2.7 venv using a Python 3 interpreter. You need Python 3.10 to create a Python 3.10 venv.

There are plenty of situations where the Python interpreter version matters. As a non-exhaustive list, you have libraries that compile code, non-Python languages that link to cpython, build scripts that do different things depending on wheel/setup/other-bundled-stuff, Python code that uses removed compat shims like importlib-metadata...

If you haven't run into one of those situations yet, congratulations. I've been through this already, and making a reproducible environment does require first installing a pinned version of the Python interpreter and THEN setting up a venv using that particular interpreter.

[zahlman]

> You need Python 3.10 to create a Python 3.10 venv.

Yes, but this doesn't need to cause a problem for those of us using bare-bones tooling. Speaking for myself, I just... run venv with that version of Python. Because I have a global installation of everything from 3.5 to 3.13 inclusive, plus 2.7 (just so I can verify what I post online about how things used to work in 2.x). And I never install anything to my base Python versions, because my system Python is Apt's territory and the ones I built from source are only and specifically there to make venvs with.

Compiling Python from source on Linux is about as straightforward as compiling from source ever gets, honestly. (The Python dev guide has some useful tips - https://devguide.python.org/getting-started/setup-building/i... .)

[camdenreslink]

This seems like a solved problem with pyenv which is very popular. You can also include a `.python-version` in your git repo root dir to automatically use the correct python interpreter version when in the scope of that repo.

[nyrikki]

Most OS's use package managers these days, expecting ANY language to not run into major problems with that is just asking for trouble.

Remember that Version SAT is NP-complete: https://research.swtch.com/version-sat

How do you expect to support violating that contract, where the OS vendor, package maintainers issue updates and security fixes, while you maintain your code, with the realization that Python releases have two years of bug fixes and three additional years of security fixes. Are you actually forced into this model or are you just adding to code debt refers and adding to the accumulation of poorly written or unmaintainable code that makes future changes even harder than they are today?

Using OS package managers helps you avoid accumulating so much code debt, making moving to newer versions the easy path actually results in better written and maintainable code.

If you just listen to depreciation warnings, and prioritize trying to stay close to the current released versions, and insisting that running on unsupported versions is an incident and not hiding it under the rug, things get better over time.

That said, the Python direct download paths are very stable.

3.13.0 being: www.python.org/downloads/release/python-3130/ 2.5.5 being: www.python.org/downloads/release/python-255/

I would still recommend creating packages from those downloads, while there are possibly better options, fpm has treated me well for well over a decade.

Looking in homebrew, which is probably one of the bigger unknowns, they have targets for everything going back to 3.8, with all major Linux distros supporting farther back and obviously VM/containers are an option there too.

  brew info --json python3 | jq -r '.[].versioned_formulae[]'
  python@3.12
  python@3.11
  python@3.10
  python@3.9
  python@3.8

I have run into the situations above, but the use of unsupported and/or end-of-life software is not something you intentionally help an organization do.

Especially with the actions by several agencies to bust EULA's

While IANAL we will see what happens in the courts, when the CISA labels something as "This dangerous practice is especially egregious in technologies accessible from the Internet"

There is a very big difference between sustainment needs and active development, one is a reality, the other could possibly be framed as "Gross negligence", invalidating any EULA protections in many parts of the country.

Obviously if you are running other peoples software the calculus of trade-offs changes.

But I have done the Python 2.7 to 3.x migration for several open source and commercial projects in the past. I can promise you that the time you waste on using 2.7 with software you control is far more expensive than updating.

If you are stuck on 2.7 on a product under active development, which has been EOL for 5 years and was warned to be going away for almost a decade prior to that...the problem you have is not with any 3rd party dependancies, the problem is with your organization.

No 3rd party vendor solution will fix those problems for you.

[merb]

pyproject.toml solves this nowadays

[rcxdude]

This is more or less my experience, but I think in part it took a while for pip to actually get into a usable position, hence some of the proliferation of other options.

[thingification]

That might be fine in your context. People's problems are real, though. What they're almost always missing is separating the source code from the compiled output ("lock files"). Pick a tool to help with that, commit both files to your ("one's") project, problem solved.

People end up committing either one or the other, not both, but:

- You need the source code, else your project is hard to update ("why did they pick these versions exactly?" - the answer is the source code).

- You need the compiled pinned versions in the lock file, else if dependencies are complicated or fast-moving or a project goes unmaintained, installing it becomes a huge mindless boring timesink (hello machine learning, all three counts).

Whenever I see people complaining about python dependencies, most of the time it seems just that somebody lacked this concept, or didn't know how to do it with python, or are put off by too many choices? That plus that ML projects are moving quickly and may have heavy "system" dependencies (CUDA).

[thingification]

To be more concrete:

In the source code - e.g. requirements.in (in the case of pip-tools or uv's clone of that: uv pip compile + uv pip sync), one lists the names of the projects one's application depends on, with a few version constraints explained with comments (`someproject <= 5.3 # right now spamalyzer doesn't seem to work with 5.4`).

In the compiled output - i.e. the lock files (pip-tools or uv pip sync/compile use requirements.txt for this) one makes sure every version is pinned to one specific version, to form a set of versions that work together. A tool (like uv pip compile) will generate the lock files from the source code, picking versions that are declared (in PyPI metadata) should work together.

My advice: pip-tools (pip-compile + pip-sync) does this very nicely - even better, uv's clone of pip-tools (uv pip compile + uv pip sync), which runs faster. Goes nicely with:

- pyproject.toml (project config / metadata)

- plain old setuptools (works fine, doesn't change: great)

- requirements.in: the source for pip-tools (that's all pip-tools does: great! uv has a faster clone)

- pyenv to install python versions for you (that's all it does: great! again uv has a faster clone)

- virtualenv to make separate sandboxed sets of installed python libraries (that's all it does: great! again uv has a faster clone)

- maybe a few tiny bash scripts, maybe a Makefile or similar just as a way to list out some canned commands

- actually write down the commands you run in your README

PS: the point of `uv pip sync` over `uv pip install -r requirements.txt` is that the former will uninstall packages that aren't explicitly listed in requirements.txt.

uv also has a poetry-like do-everything 'managed' everything-is-glued-together framework (OK you can see my bias). Personally I don't understand the benefits of that over its nice re-implementations of existing unix-y tools, except I guess for popularizing python lockfiles - but can't we just market the idea "lock your versions"? The idea is the good part!

[gkhartman]

That's been my experience too. The main complaint I hear about this workflow is that venvs can't be moved without breaking. I just rebuild my venv in each new new location, but that rebuild time can add up for projects with many large scientific packages. Uv solved that pain point for me, since it provides a "pip install" implementation that runs in a fraction of the time.

[mrbungie]

This. And also try always to fix the version of the requirements, and that's it.

Never had a problem making reproducible builds doing so.

[Fethbita]

I had issues with exactly this method. One of my dependencies was pulled off to a paid model so my project no longer worked.

[mrbungie]

Anyone remember the leftpad fiasco in the node ecosystem? That could happen in any dependency system that allows owners to unpublish dependencies and that's one risk users must weigh when adding them.

[Alir3z4]

Yeah, I assume pinning the version is something everyone does? Or probably many just don't and will have those "python deps management is a mess drama".

TBH, I've seen tutorials or even some companies simply do `pip freeze > requirements.txt` :shrug: which is a mess.

[atoav]

Then you deploy to an old debian and everything falls apart.

[Alir3z4]

Not really.

`pyproject.toml` let's you set the min python version. If not met, it won't install.

Regardless, majority of the times, deployment is done via Docker.

[atoav]

I was reacting to a comment that said a dependencies.txt and a venv was enough, so in the model I critisized there is no pyproject.toml

> Regardless, majority of the times, deployment is done via Docker.

What, generally? In your peer circle? I'd say [citation needed] — docker has the problematic habit of inducing more moveable parts that can bite you, so I know many who — if given the choice — would rather deploy python projects without it. Having deployed many python application on actual bare metal and VMs alike I'd say the ratio of Docker VS just Python is more like 1:8.