This makes me so happy. Back when we had Jenkins slaves, one of our devops guys set a pipeline up that pip installed different versions over the top of system packages causing weird intermittent failures everywhere. Different pipelines would be running in different requirements files. I revoked sudo privs immediately for Jenkins (I didn't add them in the first place) and reprovisioned the whole build cluster resulting in pipelines breaking consistently where they should have been breaking: trying to do stupid stuff.
Personally I only ever use the system python packages on Linux if I can get away with it. Saves a whole world of problems.
> Back when we had Jenkins slaves, one of our devops guys set a pipeline up that pip installed different versions over the top of system packages causing weird intermittent failures everywhere.
Not everyone might like containers, but using them for CI seems like a good way to avoid situations like this, at least when viable (e.g. web development). You get to choose what container images you need for your build, do whatever is necessary inside of them and they're essentially thrown away after once you're done with what you need to do, cache aside. They also don't have any significant impact or dependencies on the system configuration either, as long as you have some sort of a supported container runtime.
Second containers (however controversial they may otherwise be). The official python Alpine container is < 20mb and the slim Debian variants are < 45mb. For many of my Python projects I end up needing to install various system dependency libs like CUDA, libav, libsndfile, etc. I tend to be a "fan" of containers generally and containers seem like the best way to handle situations like mine.
Nowhere in the official Python documentation (where 99% of new python users are going to go) does it warn or even talk about Linux and Debian specific issues like only using apt packaged versions of dependencies. It wasn't even until recent years that pip gave a hint or warning something might break in those setups. The situation with Python on Debian has been pretty bad IMHO with a cloistered group of people saying the status quo is just fine because it works for them exclusively.
"The situation with Python on Debian has been pretty bad IMHO"
The situation with anything has been pretty bad in Debian lately.
I'm all for the minimalistic approach in regards to Python. It's Ok to provide only the packages needed by applications and the core system. For everything else, there's pip.
EDIT:
I've meant to say, there's pip inside a venv.
I agree, the problem is if you just 'pip install foo' on Debian, like 99% of python docs and readmes say, it instantly fails. Worse yet if you think you should just elevate to root and force pip install to work you can potentially break the Python install. It's a nasty footgun.
Mixing pip with another package manager has always seemed weird to me. You're just asking for things to conflict and break.
I noticed with Homebrew that there was no way to untangle packages installed through pip and ones installed through Homebrew. After dealing with that mess once, I now make sure to use pip install --user. It can still cause things to break, but if that does happen it's at least easy to nuke the packages installed to my home directory.
Good. Now we just need to get pip itself updated so it refuses to run outside of a venv, and refuses to run unless invoked with "python -m pip" and we'll finally have something at least half decent.
And don't even get me started about how much better npm is at publishing packages, versus pip's refusal to add the same user friendliness.
PEP 704 is a recent proposal to require a virtual environment by default for any package installer - https://peps.python.org/pep-0704/. Again, you can opt-out if you want.
You could have a container whose entry point is a shell script that calls multiple Python programs that need different environments, or a multiprocess container that runs multiple Python programs, although I guess you could still address either by breaking down your containers differently.
Personally I only ever use the system python packages on Linux if I can get away with it. Saves a whole world of problems.