[bbojan]

Same here. I always see people complaining about Python packaging, but rarely run into issues myself. Maybe it's an OS issue? I use Linux.

Some of the projects at work use gigabytes of computer vision dependencies and again I don't remember the last time I had any issues.

Could some of the people who do have issues give an example or two of the problems they run into?

[epage]

Part of the problem is the frustration is the curse of choice, the need to research what tool to use.

Before the new crowd of tools, like poetry, the problem was also having to figure out how to chain all the scattered tools together. I likely still have some bias from this experience.

I stopped doing python before getting to adopt any of these tools but when I researched Poetry for a company to use, a fatal flaw is that they copied too much from Rust/Cargo, Cargo was born with its community and they help shaped each other. The rust community has a strong adherance to semver. In Python, you have at mess of versioning schemes (e.g. CalVer) and low quality version requirements. You need a way to override version requirements but they refuse.

Doing cross platform development? The tools that do locking today via `requirements.txt` generate platform-specific lockfiles.

[oblio]

You don't try to distribute Python applications to multiple OSes, I assume?

[throwup238]

> CUDA version on your system does not match the CUDA version torch was compiled with

Bane of my existence.

[Shish2k]

Yes this - I’ve been using python for decades, and been mostly fine (dropping down to ~zero packaging & deployment issues since I started putting all my services into docker images). But AI stuff? Somehow that manages to be a massive pain in the ass every single time

[gipp]

I mostly agree with the GP comment, but CUDA support has definitely been the one exception that was just an absolutely infuriating experience.

[marbs]

Here are the two main packaging issues I run into, specifically when using Poetry:

1) Lack of support for building extension modules (as mentioned by the article). There is a workaround using an undocumented feature [0], which I've tried, but ultimately decided it was not the right approach. I still use Poetry, but build the extension as a separate step in CI, rather than kludging it into Poetry.

2) Lack of support for offline installs [1], e.g. being able to download the dependencies, copy them to another machine, and perform the install from the downloaded dependencies (similar to using "pip --no-index --find-links=."). Again, you can work around this (by using "poetry export --with-credentials" and "pip download" for fetching the dependencies, then firing up pypiserver [2] to run a local PyPI server on the offline machine), but ideally this would all be a first class feature of Poetry, similar to how it is in pip.

I don't have the capacity to create Pull Requests for addressing these issues with Poetry, and I'm very grateful for the maintainers and those who do contribute. Instead, on the linked issues I share my notes on the matter, in the hope that it may at least help others and potentially get us closer to a solution.

Regardless, I'm sticking with Poetry for now. Though to be fair, the only other Python packaging tools I've used extensively are Pipenv and pip/setuptools. It's time consuming to thoroughly try out these other packaging tools, and is generally lower priority than developing features/fixing bugs, so it's helpful to read about the author's experience with these other tools, such as PDM and Hatch.

[0] https://github.com/python-poetry/poetry/issues/2740

[1] https://github.com/python-poetry/poetry/issues/2184

[2] https://pypi.org/project/pypiserver/

[coldtea]

>I always see people complaining about Python packaging, but rarely run into issues myself. Maybe it's an OS issue? I use Linux.

No, it's a "what you do with it issue". Not necessarily about the mere "number of dependencies" used, when e.g. someone just makes some conda env and is fine with it.

Things like relocating, provisioning, reproducibility, version updating, cross platform, etc, all have their issues, and it gets worse when you need to build your own packages.