|
I would say that one important factor is the tooling. Instead of having one or two standard tools, as seen in Java (maven/gradle), Node (npm/yarn), .NET (dotnet CLI and MSBuild/maybe FAKE), Python has a plethora of opinionated tools. venv+pip is built-in, but doesn’t handle any of the “where to put the venv” parts, and doesn’t handle making packages installable (and the classic solution is setuptools). There are many competing projects, like pipenv (which does not work at all for libraries), poetry (which uses a non-standard way of specifying metadata and dependencies, and isn’t too friendly with the rest of the packaging world), flit, hatch, etc. Why are there so many competing tools? Why can’t the community just pick one and make it a good tool for all use-cases? There’s an organization called “Python Packaging Authority”, and it maintains almost all tools mentioned in the previous paragraph (except the stdlib venv, and poetry). If they’re an authority, they should just say “___ is the way to go, we’ll add the missing use-cases to it, here’s a tool to migrate everything else to ___”. Instead, they have done things that support the proliferation of tools, like the PEP 517 (which is a standard API for package managers to talk to build tools). Another deficiency in the Python package management system is the fact that you need virtual environments to get things done, and that they are often finicky. Some of the modern tools hide the venv from you, but this gets less practical if you’re trying to use things from within scripts, or trying to point your WSGI server at the venv, or in multi-user scenarios. Some of the modern tools put it in .venv and manage it for you, but venvs can randomly break in cases the tool might not be aware of (eg. in some system package managers, upgrading Python to a new minor version would cause venvs to break, because symlink targets moved.) Node solves this by having a `node_modules` directory. .NET does stuff at build-time that most people don’t need to care about, and then just puts the .dll files for your dependencies next to the .dll with your thing, and the loader looks in the directory your code came from. Python has a proposal for `__pypackages__` [0], but it’s been there since May 2018 with not much progress. [0] https://peps.python.org/pep-0582/ |
One, Python and its packaging story is old (remember, Python predates Linux). That has given folks plenty of time to either come up with their own solutions since Python predates widespread internet usage (PyPI has not always been around).
Two, a lack of standards. Tying back into the "old" point, not everything was initially designed. There has been work to chip away at this and get more standards behind things, but getting folks to update their packaging code is *hard*; most people copy their packaging code from their last project and don't really try to update it. Getting changes to propagate through an ecosystem as large as Python's takes years; a decade is the typical time frame considered for complete uptake.
Three, the lack of standards means tools come up with their own solution which then isn't compatible with anyone else. That means when someone innovates, it can very quickly get locked up behind a single tool. That means folks end up reinventing the wheel for various reasons (e.g. lock files). Different approaches leads to different opinions, which leads to folks choosing different tools for different reasons. And when that happens, there ends up being a lack of consensus.
And four, this is almost entirely driven by volunteers with very few people paid in any way to work on this stuff (I think there might be like 2 people who contribute to pip, 2.5 folks for PyPI).
And good luck telling someone that their preferred workflow isn't the "chosen" workflow that the whole community is going to switch to. People don't like being told they are going to have to change, especially if they believe their approach is superior for whatever reason. Multiply that by the size of the Python community and you can see the pitchforks quite clearly on the horizon.
Now that isn't to say work isn't being done to improve the situation. The PyPA side of packaging has been working on standards for quite some time and is getting some traction with them (e.g. pyproject.toml is a great example of that). But we do have some more standards to work out. We are also regularly discussing how to come up with some singular tool/UX that people can get behind for most use cases, but see the above comments about the size of a challenge that it is and thus why it hasn't happened yet. But people are aware and trying to figure all of this stuff out.