[patagonia]

Imagine I’m just getting started with Python, and I see this article. I think to myself, “Awesome, a primer!”

Then I start reading (these comments)... mayyybe I should try Julia... or anything else, at least while I’m still getting started.

[jefurii]

Just use pip and virtualenv. I've been using them on a cluster of related projects for years and they've been rock-solid.

[mixmastamyk]

And you can avoid virtualenvs as well until you need to ship a large app.

"pip install --user pkg" is good enough for beginners and writing libraries with few deps.

[Serow225]

Does anyone know why the Python community has seemed to struggle with package mgmt fragmentation/churn so much over the years, compared to other languages? Did Guido just not really care about package mgmt?

[xapata]

Guido wrote Python in 1990. Did you use a package manager back then? I didn't.

Part of the issue is how well Python integrates with non-Python dependencies. Before conda, when I wanted to upgrade some Python projects, I'd get errors complaining about my Fortran compiler. These days, most of the major projects upload precompiled binaries for major platforms to PyPI, but when it was just source code...

[Serow225]

Sure, but couldn't it have been given some BFDL/high-level leadership importance in the last five years to rein in the craziness?

[jonafato]

The PyPA team has done a lot over the past five years. The changelog for pip (https://pip.pypa.io/en/stable/news/) contains quite a bit, PyPI was migrated to Warehouse, and there have been several PEPs focused on improving the packaging situation. A lot of these ideas come from various people in the community and get formalized as official recommendations or tools, but these things take time, especially accounting for backward compatibility in an ecosystem as large and mature as Python's.

The short answer to "why isn't this solved?" is "it's hard, and there's a lot to do". Development practices change over time, and the tooling continues to evolve with them. It's easy to see a broad survey like this and think that there's too much going on, but taken at a high level, the space is definitely trending in the right direction.

(Note: I'm not part of the PyPA, but I'm interested in this area and try to follow along from the outside.)

[Serow225]

Understood, I guess I'm wondering why it hasn't been possible to cull more of the less-successful attempts, or at least make it obvious to newer users what is legacy. As an outsider/newer person to Python, the number of package mgmt options to consider is vast and confusing, it would be helpful if there was one (or a few) more "blessed" solutions :)

[toyg]

> the number of package mgmt options to consider is vast and confusing

Part of the issue is due to the success of Python in very different niches. The likes of Rails or Node can concentrate on specific ecosystems, which account for the bulk of their users and have a limited set of scenarios they have to support; whereas Python users come from sysadmin to data-crunching to web to desktop development to games to to to...

So each packaging tool comes with certain ideas, usually a result of the author's experience; maybe they work very well in this or that scenario, but then they break badly on others and sizeable chunks of the community revolt. So a new tool comes around and the cycle starts again, but now people also want compatibility with the old tool.

I suspect part of the solution will require splits between niches. It already happened with Anaconda, which has basically become the standard in a particular group of users (academia / datascience). Since that came around, lamentations around building C libraries have substantially reduced (to be fair, the arrival of precompiled wheels on PyPI also helped). Some similarly-specialized tool might eventually emerge as standard for other niches.

Python developers are cats and they are pretty hard to herd at the best of times, which is unsurprising -- who would stick around a language that is almost 30 years old and was never promoted by any major vendor? Only hard-headed fools like myself.

[jonafato]

There are some "blessed" recommendations at https://packaging.python.org/guides/tool-recommendations/ (the Python Packaging Authority is about as official as you're going to get), but this boils down to it being a large open source community. No one's going to cull other people's efforts, but tools do merge on occasion (e.g. the functionality of https://github.com/erikrose/peep has been merged into pip, so peep is deprecated now).

[crad]

Problem is like that XKCD comic about standards. People keep reinventing the wheel instead of embracing and evolving the one thing that works well and has existed for a long time.

[paulie_a]

No it is dead simple. Requirements separated by environment and a virtualenv. It's awesome compared to JavaScript

[zmmmmm]

Wait until someone from the python community recites to you "there should be one and only one good way to do something" ....!

[jonafato]

I think plenty of people in the Python community will earnestly say that while acknowledging that there isn't universal agreement on what that one good way is. It's an ideal to strive for, not a statement of fact.

[zitterbewegung]

This isn’t a primer but more like a survey of packaging options.

Using pip and virtualenv is usually fine or using pipenv .

[patagonia]

“Usually” is great until it’s not. I don’t want to get invested in a languages only to discover issues down the line. So, if something as basic as packaging is potentially problematic and there are other options available, I might look elsewhere before rolling the dice that this problem is not too problematic.

Perhaps “Primer” was the wrong word, but I believe the sentiment is valid. Reading the comments, there is simply no consensus. If code readablility is important since code will be read more than written, packing is important because in many many scenarios that count code will be distributed more than it will be read.

It is simply frustrating that the typical response to comments like my original comment is some form of “it’s not as bad as you think”. Look, we have a problem here. A problem many other languages deem important enough to solve upfront. It’s been a problem for a long long time.

[aprdm]

pip and virtualenv has been solving the problem and is the standard for ever.

other people tried different approaches, like, pipenv and is just that, a separate project trying to solve the same problem.

I don't know why / who said that pipenv is the official recommended way, if it is it should not be and I hope it is not.

[craftsman]

> I don't know why / who said that pipenv is the official recommended way, if it is it should not be and I hope it is not.

The Python Packaging Authority says that pipenv is the first "recommended way" for managing application dependencies: https://packaging.python.org/guides/tool-recommendations/#ap...

Except if pipenv doesn't meet your needs. Then use pip.

Or, if you need cross-platform support, use buildout.

Or, if you are doing scientific computation, don't use any of those, use conda, Hashdist, or Spack.

Or if you need to create a package, use setuptools and twine.

So, no, pip and virtualenv don't solve the problem, because there are a lot of different problems and use-cases. I can say from my experience that conda is _the_ best solution to the problem for scientific work.

[xapata]

The Python community is not monolithic. PyPA has made some decisions that others might disagree with.

[vxNsr]

Kinda exactly what went through my mind, I've dabbled in python, but only to use other ppl's projects, which sometimes required setting up pip, etc. So I was looking forward to reading about how it'd been simplified :(

[xapata]

https://www.anaconda.com/download/