[blame-troi]

Every time I get the urge to pick up Python, I find the packaging situation so off putting that I quickly find some other rabbit hole to dive into. I find emacs and vim add on configuration easy and transparent by comparison.

[andrewflnr]

As OP suggests, you can ignore basically all of it. If you're just messing around, use your system Python. That'll get you a long way. If you're Starting A Project and want to install packages locally, use the virtualenv package that came with your system Python. That's gotten me through basically all my Python programming for 15 years. Anything more complicated than that, you can learn when you come to it.

[nerpderp82]

> use your system Python.

Maybe you are an accidental accelerationist? Using the system Python is fastest way to a broken system and a broken Python.

[kaashif]

What? Why? How?

I always use the system Python plus virtualenv and that works fine, it's certainly never broken my system...

I don't even know by what mechanism that breakage would happen.

[w0m]

'sudo pip install <something huge with a ton of deps>' could potentially make system packages go wonky. Working inside a venv is perfectly safe.

[kaashif]

Oh, I guess I never even considered doing that for some reason.

If you just run "pip install" doesn't literally tell you to use the --user flag or virtualenv?

I guess it's probably an easy mistake to make.

[nerpderp82]

If you are using system python only to create venvs you are fine. As soon as you install or start modifying the system python in anyway, you are on the path to ruin.

[unmole]

Somehow in the 15 years of working with Python on Linux, I've never broken anything using pip.

[sigjuice]

Likely due to your good taste and judgement of never ever running pip with sudo.

[integricho]

I'd argue that running pip or any package installation with sudo is the devs fault.

[salawat]

Never tried to update something eh?

Debian is particularly bad because they'd comingle python 2 and 3 packages in the same filesystem locations if you used pip out of the box.

That was a horrid discovery on my part. I assumed, coming from java where this sort of thing has been a long solved problem, that no one would be daft enough to do something like that.

[Filligree]

God save people who don't know you're 'supposed' to use something else.

Most projects state something like "run pip install whatever", which people will then do. If you're lucky, it will ask for sudo. If you're even luckier, you stop to think before entering your password.

[nerpderp82]

I have a default Python env that my shell sources for every login shell. I can deactivate it and activate different envs on a per project basis.

That default Python env means I don't ever modify the system python for any reason. I type `python` and I get my own Python3 from the base.env

If you use homebrew, there are two settings that help keep your old python binaries around so that venvs that reference those old envs don't break.

HOMEBREW_NO_CLEANUP_FORMULAE, this takes a lists of packages to not cleanup

HOMEBREW_NO_INSTALL_CLEANUP, prevents old packages from getting removed

[deaddodo]

Node has the exact same issue for people who globally install all packages. The only differences are A) python makes environment management the dev's choice/responsibility and B) defaults to system vs environmental.

I actually far prefer the Python method to, say, Go's GOPATH. But, I far prefer Rust/Cargo to either.

[unscaled]

Sure, GOPATH really sucked, but that's why we got Go Modules. I don't think it's fair to compare Python to Go anymore.

I wish Python got a standard integrated solution for package management that works.

pip doesn't cut it in my book since it doesn't let you specify dependencies reproducibly. It either doesn't support lockfiles or ONLY supports having a lockfile, without dependencies.

venv also doesn't cut it since you have to remember to explicitly activate it and keep track of which venv is activated right now.

If we take a look at the much-maligned Node.js+npm, it's still far better than what built-in tools in Python let you have. Yes, Node.js doesn't provide full isolation from globally-installed node_modules, but at least it supports a local node_modules directory and lets it take preference. Notably Node.js+npm, with all its warts, doesn't suffer from the the two issues I've mentioned above.

[noloblo]

Golang is far better when it comes to go path and compile speeds

[dns_snek]

> If you're Starting A Project and want to install packages locally, use the virtualenv package that came with your system Python.

I think this should be emphasized more, I don't think it's just a matter of preference, not separating packages between projects using virtualenvs will land you in a world of hurt as soon as you want to update or uninstall any of them and they're hopelessly entangled with system packages and other projects.

[andrewflnr]

If you're just starting with the language though, you might only have one project for a long time, or your projects might have disjoint dependencies (or close enough; I'm imagining someone with both a Django and pygame project). And you can learn a lot without any libraries at all. As a teen, I went a couple years and learned a lot before I learned how to use virtualenv. You'll want it sooner, but it absolutely shouldn't be a barrier to getting your feet wet.

[dns_snek]

That's certainly valid, if someone is just getting started like you say then it's not something they should worry about. I wasn't thinking of complete beginners when I wrote that comment.

[hodgesrm]

Exactly the same thing happened to me recently. I found refuge in Golang. Not my favorite language but it's tough to beat for utilities that you want to distribute quickly.

[nonethewiser]

For all the shit node gets for being too dependency heavy, npm, yarn, pnpm all work exactly how I’d expect.

[hodgesrm]

I love npm. It actually works. ;)

[noloblo]

Not to mention golang sub second compile time and all the productivity boosts coming from that

[pjmlp]

Hardly impressive for anyone that has used compiled languages with modules since the 1980's.

What is impressive, is how so many in the industry have forgotten about them.

[noloblo]

With all these years of improving compile times why is haskell compiled time very slow still?

[hodgesrm]

Incremental compilation has been a thing since the days of Smalltalk. (At least.) It's a feature of every Java IDE I've ever used. I do appreciate quick Golang builds but being able to change something in an IDE and see instantly if it compiles is a big plus of many lanugage.s

[nonethewiser]

Install dependency.

Save in requirements.txt. Includes sub dependencies that are system specific.

Install from requirements.txt on different machine and get errors.

Uninstall dependency and save to requirements.txt.

Look at requirements.txt and see that sub decencies are still there.

What is the right way to avoid these issues on Python?

[bart_spoon]

I use pip-compile in the pip-tools package:

Keep your high-level dependencies in requirements.in.

Create a virtual environment using venv.

Run `pip-compile resolver=backtracking`. It autogenerates a requirements.txt file with all dependencies and sub-dependencies and their versions. This essentially acts as a lock file from other languages/frameworks.

Install from the autogenerated requirements.txt file in the venv virtual environment.

If a dependency changes, change in requirements.in, recompile the .txt file, and reinstall.

[MonaroVXR]

>Run `pip-compile resolver=backtracking`

This is new for me. I used `pip freeze > requirements.txt`

[Fradow]

My solution is to:

* use a virtualenv, so that dependencies are installed per project and not globally

* only specify my top-level dependencies in a requirements.in file, and let pip-compile (a dependency, part of pip-tools) compile my requirements.txt

So far it has served me well and I've not encountered any error due to this method.

I cannot claim it is "the right way" though, just something that works for me.

[mikepurvis]

Specify your actual dependencies only in setup.py/cfg and only ever write your requirements file from pip freeze.

Or junk all this and just use poetry, which manages both the abstract dependencies (pyproject.toml) and concrete ones (poetry.lock).

[jakewins]

Poetry seems to solve the dependency spec part, both for libraries that need to ship specs for supported version ranges of dependencies and apps that ship lock files.

However: poetry still falls short in managing the python runtime, I am continuously having to divert time to help our data scientists untangle the messes poetry makes with virtualenvs and their local python setups.

Also, they broke backwards compat on the lock file format? So now devs running newer poetry versions break projects for devs on older versions because the lock files aren’t compatible?!

[mikepurvis]

Yes, I was very upset about the lockfile format change when we hit that; seemed like a very shortsighted thing to have done, but I guess I don't know the actual motivations or what was being achieved with it.

[jwestbury]

Use pipreqs instead of pip freeze. It resolves the minimal set of dependencies for your package.

[MonaroVXR]

Going from Windows to Linux wasn't a problem for me

[liveoneggs]

containers

[influx]

I use docker and just use pip install requirements.txt. Been working well for me.