so kind of allowing pull images from nix store, mounting shared host nix store per node into each container, incremental fast rebuilds, generating basic pod configs are good things.
and local, ci and remote runs same flows and envs.
Jotting down a few quick thoughts here but we can totally go deep.
This is something Michael Brantley started working on a few months ago to test out how to make it super easy to ease and leverage existing Nix & Flox architecture.
One of the core differences from my quick perspective is that it specifically leverages the unique way that Flox environments are rendered without performing a nix evaluation, making it safe and optimally performant for the k8s node to realize the packages directly on the node, outside of a container.
Wrong. If you know nix then you know "leverages the unique way that Flox environments are rendered without performing a nix evaluation" is a very significant statement.
Yeah, it's essentially cached eval, the key being where/how that eval is stored.
When you create a Flox environment, we evaluate the Nix expressions once and store the concrete result (ie exact store paths) in FloxHub. The k8s node just fetches that pre-rendered manifest and bind-mounts the packages with no evaluation step at pod startup.
It's like the difference between giving the node a recipe to interpret vs. giving it a shopping list of exact items. Faster, safer, and the node doesn't need to know how to cook (evaluate Nix). I don't know, there's a metaphor here somewhere, I'll find it.
Only so much room for magic, for sure, but tons of room for efficiency and optimization.
Correction: we don't eval when you create environments.
Our catalog continuously pre-evaluates nixpkgs in the background. 'flox install' just selects from pre-evaluated packages -- no eval needed, ever. The k8s node fetches the manifest and mounts the packages.
Eval is done once, centrally, continuously. So... even more pre-val'd, so to speak.
https://github.com/pdtpartners/nix-snapshotter
so kind of allowing pull images from nix store, mounting shared host nix store per node into each container, incremental fast rebuilds, generating basic pod configs are good things.
and local, ci and remote runs same flows and envs.