[mikepurvis]

From experience just maintaining a private 1000+ package overlay to nixpkgs, this sounds super fraught, at least for any of the scenarios most typically interesting to developers, like wanting a backport of some library version that just landed to your otherwise-stable months old pin of nixpkgs.

For a lot of the most important ones, though, nixpkgs already maintains multiple recent versions, though— fifteen boosts in there right now: https://search.nixos.org/packages?channel=unstable&from=0&si...

[methyl]

Yeah I found it odd that there is no first class feature for versioning. I wonder what’s the reason, since it’s a pretty obvious need for a package manager.

[pxc]

> Yeah I found it odd that there is no first class feature for versioning. I wonder what’s the reason, since it’s a pretty obvious need for a package manager.

Binary package managers need a first-class notion of versioning for install-time dependency resolution, Nix doesn't actually perform install-time dependency resolution.

Nixpkgs is a monorepo, and thanks to the way it supports side-by-side installations of conflicting versions, instead of using version numbers to check/assert compatibility, Nix just ships each package with exactly the dependencies it was built with, all the way down the dependency closure.

While it doesn't retain that fixity all the way down the dependency chain in the case of 'partial upgrades', this is also how Homebrew works. Gentoo Portage does include some functionality for reasoning about version constraints. (I'm not sure about other source-based package management systems like Ravenports or FreeBSD Ports or MacPorts.)

If Nixpkgs (or a counterpart, like an archive repo) included not just the 'current' set of combinations but a collection of old build recipes along with a record of version numbers for which builds had been successful with that recipe, or a range of versions that were known/expected to work... that could be really useful for projects downstream in the Nix ecosystem. It could power tools that import or generate Nix code given some packages and version constraints. But it's not actually needed in order for Nix to match the practical functionality of best-in-class binary package management systems like dnf.

There actually are already some tools in the Nix ecosystem that use (indices of?) old copies of Nixpkgs as this kind of archive: mach-nix comes to mind! Idr if it uses them at a source level for creating new build definitions or if it actually uses the whole dependency tree, embedded in those old versions of Nixpkgs. Either way, a better version of that is what I think Nixpkgs could soundly facilitate, while still only maintaining a single combination of packages as the official, coherent, cached collection for use by end users.

[mikepurvis]

Based on the amount of patching and such that is necessary in many packages, plus the constraint of not linking two different versions of libXYZ into your final binary, I'd expect it would be pretty hard to have a truly unified environment of multiple nixpkgs pins, at least without a facility for maintaining another big patching layer.

[pxc]

You'd want to be using old Nixpkgs definitions as a starting point for importing a package definition that will work with a package's build system at the given version, and trying to rebuild it against the libs that are already in your environment.

You could do the same basic versioned dependency solving all the way down your dependency tree using the same tool, whenever one of the recipes expected a version not in your current environment.

That's what it would mean to incorporate real dependency solving into Nix/Nixpkgs. I don't think you'd want packages in Nixpkgs' top-level/all-packages to rely on that feature, but you could have it be there for devs who want to build out bespoke environments but rely on the collective wisdom about how to build this and that at such-and-such version currently encoded in the history of Nixpkgs.

Sometimes you couldn't be offered a solution that you're confident would work— this kind of thing would be a starting point for maintainers of complex environments. But it would make porting old versions forward to a newer env easier just by helping automate that process of digging for examples in the history of Nixpkgs.

Maybe this is basically what you have in mind with the idea of a 'patching layer'. In that case, I agree that a really useful implementatiom would involve tackling some substantial problems. But I do think it is possible to make something useful along those lines, and it would be an exciting tool even if it was only 60% of the way there. Hence my curiosity a couple of posts up. :)

[mikepurvis]

I might just lack imagination, but I think the wheels are already well in motion for at least one significant piece, which is many more upstream maintainers having some awareness of Nix and Guix (and now Tea too, I guess) and understanding the value of things like reproducibility, relocateability, not accessing the network during build, not making assumptions about where stuff will be located or that the different build outputs (lib/bin, dev, doc, whatever) may end up in fully separate trees. This goes doubly so for tools maintainers like the people at Kitware, who wield an enormous amount of power in steering devs toward patterns that allow software to be packaged for Nix with minimal patching.

All this lowers the fragility of the package definition and I think helps increase the likelihood of that software being forward- and backward-compatible with different versions of its underlying dependency pool.

[visarga]

This complicated mess makes me think maybe language models like Codex could be trained to solve any configuration request by learning from trial and error.