Hacker News new | ask | show | jobs
by alextgordon 4377 days ago
Ha! There is nothing well-designed about the mutually incompatible, political hell that is OS package managers.

The best solution (as taken by npm, virtualenv and others) is to install libraries locally to the project that is building them.

That way, package management becomes the sole concern of the build system.

"Accumulation" is a good thing, it means each project has the exact version of a package that it was tested with, not some later version that a sysadmin decides it "probably should work with".

2 comments

OS package managers make two assumptions:

- that packages follow semver

- that the OS packagers are in a better position to test package combinations.

If the author releases a new version of libfoo, and A, B and C in an OS repo depend on libfoo, then the OS packagers do not release a new version of libfoo until the tests for A, B & C pass.

These are two good assumptions, and the language package world would be in much better shape if they followed those assumptions too.

Which OS are "OS package managers"? There is a world out there greater than GNU/Linux.
Cargo assumes the former.
Which is why we have testing and staging environments to make these changes on first. Conversely, when everything is pulling in their own version of a library, you run into situations like the zlib buffer overflow, where you had huge numbers of programs that needed to be rebuilt because no one used system-packaged libraries. Obsolete and vulnerable software is a liability, and not having tools which can readily tell what software needs to be updated makes quite a few peoples' jobs quite a bit harder.
You can't have it all.

Either you rely on system librarys OR every binary / library pulls in its own copy of its dependencies (npm model).

The latter lets you have multiple versions of the same library for different dependencies, without confusing your system package manager.

The former might mean 'less build time', but it's pretty much whats wrong with the C/C++ ecosystem; you can't track the system libraries from the package tool, and you get out of sync.

Pillow and freetype specifically pop to mind as an irritatingly broken recent example; when freetype upgrades, perfectly working python projects suddenly stop working because the base system freetype library has been upgraded; and the pinned python libraries that depended on the previous freetype version no longer work, because they rely on the system package manager not to do stupid things.

It would be nice if you could point cargo as a 'local binary cache' to speed things up, and make them work even if the central repository goes down; and that could be package manager friendly, I imagine.

I'd say look at the BSD ports system for an example of a system done well, and is extensible to such issues. If your app depends on projects with irresponsible developers that make unannounced API/ABI changes and the like, the general format of the ports system, and tools for maintaining private repositories like portshaker and poudriere, make it easy to create ports that let you have the best of both worlds. To add to this, one can use metapackages and the like which don't have any files in and of themselves, but will point to the latest version. So, for example, you create a port named libbar, which is always going to get you the latest and greatest, but you also have libbar-1, libbar-12, etc, that you can use when you need an explicit dependency. Additionally, there are ports of the system library -- heimdal, openssl, etc, so you can have a stable base system, but still have more recent versions around for your application. Most of the issues people have with packaging these projects seems to be much more based around inexperience with good packaging tools and practices, rather than the idea of system packaging in and of itself.
Welcome to emerge @preserved-rebuild and revdep-rebuild. Still no better way has been found to date :/.