[jonhohle]

I’m surprised all of these systems combine build logic and dependency graphs in the same config. It seems like flake might be a step in understanding these things are only tangentially related.

I would like to see composability of graphs (and other set operations on binary package repos) integrated into more dependency management systems.

FreeBSD is my server of choice and I’d love to say: create a package repo with my config package and it’s dependencies and nothing more and deploy from that knowing a million other dependencies can’t be pulled in (e.g. give me a new jail that pulls from subset).

It’s probably something I should prototype one day.

[georgyo]

I don't fully understand what you are saying, and I don't know what systems you're including in "all".

The system you describe building is exactly what nix does, as well as debian, El, and Arch. Their spec files describe both build and runtime dependices, and installing the package does not install things like Make.

Nix goes a little bit further by only including runtime dependices that it can find it the build output. It does this by scanning the output files.

But I don't understand how you could separate build graph and runtime graph. If I declare something needs foo and bar, that is useless unless I can get built foo and bar. _Something_ has to know how to build the things this hypothetical system is installing.

[jonhohle]

RPM rspecs, port Makefiles, Maven, etc. do not just define package dependencies (build, runtime, etc.) and artifacts, they also have to intimately understand how to patch, build, and package the artifacts. Making the build and package system separate from the dependency management separates concerns and allows the ability for a completely declarative “dumb” dependency graph that can be reasoned about without dealing with build logic that must be executed.

This also allows build processes to evolve without affecting the public dependency graph. This makes it easier to show dependencies are modeled and exposed correctly and makes build logic private to consumers.

[pxc]

> allows the ability for a completely declarative “dumb” dependency graph that can be reasoned about without dealing with build logic that must be executed.

Nixer Domen Kozar called this property 'static metadata' in one of his talks¹ on Python packaging from a Nix perspective years ago.

The thing he was interested in was the ability to evaluate the dependencies of an upstream software package without having to actually 'install' it or evaluate bespoke upstream code to do so. The reason for interest in this within the Nix community is that by default Nix performs builds in a restricted sandbox, and one of its restrictions is that no network access is allowed.

To use upstream build tools (e.g., Maven or Cargo or NPM, etc.) inside the Nix sandbox, then, fetching dependencies and verifying their contents is deferred from the upstream build tool to Nix, which does so in a controlled, deterministic way that just fetches and doesn't have hooks to let those deps run custom code.

In order to make that happen, Nix has to 'know' ahead of time where to fetch those dependencies and what their contents will be, and what you describe wishing for here— a 'dumb dependency graph'— is more or less exactly what Nix wants to consume (although sometimes just a list of pairs of URIs and content hashes will do). For well-behaved upstream package managers, i.e., those which can emit comprehensive static metadata, that's exactly what Nix does: it just translates a Cargo or NPM lock file into its usual conventions for describing source archives, and then that can be used to download those dependencies in the usual safe/restricted way.

For package managers that don't emit adequate 'dumb' package metadata, Nix has to proceed by either emulating and replacing those upstream dependency resolvers (very error-prone) or by a hack implemented as a Maven plugin or similar that inspects dependencies as Maven resolves them and gets the metadata Nix needs.

As for flakes, flake inputs are certainly a kind of dependency that's distinct from package dependencies in the Nix world, and also 'dumber' in the sense that flakes don't have to be 'built' like packages to be consumed by other flakes. And yeah, in the case of flakes that provide packages, you can definitely swap one flake for another and allow that new flake to provide customized build instructions if those are needed for that version of the package it provides. But there are complexities and entanglements that flakes don't/can't eliminate, since downstream dependencies can still have implicit expectations of build outputs, and you can't really know in advance if the package in the new flake will meet all of those. You kinda still have to be able to peek into those and examine them and fix them up in a pinch. Plus you can write a flake in a way that depends on the build process of a package in a flake it consumes, e.g. by using package overrides on something provided in the upstream flake.

--

1: https://youtu.be/ADSM4vR2EQ0

[jonhohle]

Yes!!

[pxc]

Awesome! I'm glad you found that context relevant. :)

It feels good to have your understanding of a problem and where different technologies might fit into better addressing it click into place like that. :D