[ahupp]

It’s not like this is unique to rust; you see similar issues with node and python. Distributions have many jobs, but one was solving the lack of package management in C. Now that every modern language a package manager, trying to apply the C package management philosophy is untenable. Specifically, the idea of a single version, globally installed, and producing distro packages for every language specific packages.

[llm_trw]

Apart from the fact building it the 'non-C way' results in a magic meat package that you have no idea what it contains: https://hpc.guix.info/blog/2021/09/whats-in-a-package/

Guix is also a distro that allows for any number of versions of the same package globally, something that language specific dependancy managers do not.

Distors are there for a reason, and anyone who doesn't understand that reason is just another contributor to the ongoing collapse of the tower of abstractions we've built.

[kpcyrd]

This is outdated information. Debian (and other distros) already had their own SBOM format called buildinfo files that encodes this kind of information.

In Debian stable ripgrep on amd64 is currently on version 13.0.0-4+b2.

The relevant buildinfo file can be found here:

https://buildinfos.debian.net/buildinfo-pool/r/rust-ripgrep/...

It encodes the entire Rust dependency graph that was used for this binary, with exact versions of each crate.

[tcfhgj]

> Distors are there for a reason

for me: make an os out of the kernel

[ahupp]

Using language-native packaging doesn't imply that you have to use binaries from wherever. In the pytorch example you can still build it as a regular part of the distribution, using the C++ dependencies/toolchain, it just means you don't try to stuff it into a versioning/distribution/install model that doesn't match the languages expectations.

[pjmlp]

Making an userspace to Linux kernel, and sorting out disagreements by creating yet another fork.

[XorNot]

Except from a management and maintenance perspective...this is a nightmare. When a security vulnerability drops somewhere, everywhere needs to be patched ASAP.

Distros (and the people who run most scales of IT org) want to be able to deploy and verify that the fix is in place - and its a huge advantage if it's a linked library that you can just deploy an upgrade for.

But if it's tons and tons of monolithic binaries, then the problem goes viral - every single one has to be recompiled, redeployed etc. And frequently at the cost of "are you only compatible with this specific revision, or was it just really easy to put that in?"

It's worth noting that docker and friends also while still suffering from this problem, don't quite suffer from it in the same way - they're shipping entire dynamically linked environments, so while not as automatic, being able to simply scan for and replace the library you know is bad is a heck of a lot easier then recompiling a statically linked exe.

People are okay with really specific dependencies when it's part of the business critical application they're supporting - i.e. the nodejs or python app which runs the business, that can do anything it wants we'll keep it running no matter what. Having this happen to the underlying distributions though?

(of note: I've run into this issue with Go - love the static deploys, but if someone finds a vulnerability in the TLS stack of Go suddenly we're rushing out rebuilds).

[ahupp]

This is conflating static linking with how the distribution handles updates. If a language is always statically linking dependencies (like Go or Rust), the distribution will have to rebuild everything that depends on a patched package whether or not they are using the language's native tools or some import into the distro package system.

What I'm specifically suggesting is:

  * Distributions package *binaries*, but not the individual libraries that those binaries depend on.
  * Distributions mirror all dependencies, so that you can (in principle) have a completely offline copy of everything that goes into the distribution.  Installing a binary uses the language-specific install tools to pull dependencies, targeting the distribution's mirror.  
  * Enough dependency tracking to know what needs to be rebuilt if there's a security update.
  * Any outside dependencies (e.g openssl) will continue to depend on whatever the distribution packages.
  * Dependencies are not globally installed, but use whatever isolation facilities the language has (so e.g, a venv for python, whatever npm does)

[XorNot]

As I see it, this all doesn't matter though as soon as "security update" enters the picture.

The problem here is upstream dev's saying "my dependency needs are absolute". And a security update ruins that: because as soon as one happens, now no matter what we're going to be replacing libraries anyway. Even your prosposal includes this: we're going to strip out openssl librares and use distro ones.

At which point everything might break anyway, because whether a security hole can be fixed at all depends on which versions of a library it affects and how. Not to mention problem's like finding the issue in one version, but it's changed enough that it's not clear whether a different version is impacted the same way.

[rat87]

Why is this an issue? Simply recompile and download each package. If the distro worries that the maintainers would take too low just fork and recompile the packages themselves. These days its really not that big of a problem in terms of disk space or network traffic. And if some packages are large its often because of images resources which can be packaged separately. It seems like a lot less effort then trying to guess if dynamicly linked library will work with every package in every case after the update.

[LtWorf]

You understand that recompiling 1 program is easier than recompiling a thousand?

[curt15]

>When a security vulnerability drops somewhere, everywhere needs to be patched ASAP.

Is that ultimately the responsibility of the application developer or the OS developer?

[XorNot]

It's "whoever turns up to do the work" but I would point out that distros generally have more people in the process who can pick up the work.

The issue is one way or another it needs to happen ASAP: so either the distro is haranguing upstream to "approve" a change, or they're going to be going in and carrying patches to Cargo.toml anyway - so this idea of "don't you dare touch my dependencies" lasts exactly as long as until you need a fix in ASAP.

[LtWorf]

Probably most of these tiny crates have 1 or 0 maintainers. Chances are that they will not be quick to fix a vulnerability.

And even if they are, for rust software that doesn't come from debian, there is no way to ensure it all gets rebuilt and updated with the fix.

Also, projects are generally slow (taking several months) to accept patches. When a distribution has fixed something and the users notice no issue, the upstream project if downloaded and compiled would be a different matter entirely.

[rlpb]

> Now that every modern language a package manager...

...they fail to integrate with dependencies written in any other language.

It's fine if you just want to sit a monoculture language software stack on top of a multilingual base platform. You can't make a functional system with one language alone, yet those who criticise distribution packaging architecture do so while simultaneously depending on this ability that language-specific package managers do not have. There is no viable alternative today. Most critics think they understand the general problem but only have narrow practical experience, so end up believing that their solution is superior while not considering the general multilingual software supply problem.

Nix isn't a solution either, because in the general case Nix isn't security-supporting arbitrary and multiple dependency versions either.