[bemeurer]

Please don't recommend that people use nix-env. It's one of Nix's biggest footguns and a huge support burden for the maintainers.

We've been actively trying to remove mentions of it from the documentation.

If you want something "installed" use home-manager. If you just want something for quick dev use nix-shell.

[mason55]

I'm so glad this is the official line now. People giving examples using nix-env -I is, IMO, one of the major reasons it's so hard to get up and running.

It's like... here are all the amazing reasons to do declarative config... and here's how you do everything using nix-env -I. Good luck figuring out how to translate it into a NixOS config so you can get all the benefits of declarative config that we just described!

[pxc]

> Good luck figuring out how to translate it into a NixOS config so you can get all the benefits of declarative config that we just described!

Guix has a command that converts an imperative profile to a declarative one. Check out the `--export-manifest` option for `guix package`: https://guix.gnu.org/manual/en/guix.html#Invoking-guix-packa...

Adding this to nix-env's replacement would be a sound approach imo. Maybe we could have one that also emits a little message for each package which is used in a NixOS module, directing users to a `programs.whatever` option instead of just dumping it in their `environment.systemPackages` list

[open-paren]

If nix-env is a big ol' footgun, are there any plans to make home-manager a part of the OS itself, instead of just a community project?

[pxc]

My hope is that when Sander van der Burg finishes drafting his RFC for mainlining his process management framework [1], which abstracts over ways to manage local services from systemd to supervisord to Docker, we can actually unify the module collection in Nixpkgs with its clones in nix-darwin and home-manager, and offer a more complete Nix experience as a configuration/service manager rather than just a package manager on non-NixOS.

Fwiw, you can already have nixos-rebuild read the home-manager configs for all of your users and deploy them as part of the normal NixOS config update process using the included NixOS module.

——

1: https://github.com/svanderburg/nix-processmgmt

[mason55]

Random nix usage question... occasionally I run into something like this, where the "install" directions are to clone a git repo and then install in an imperative way. What's the correct way to do this declaritively, e.g. in a home-manager or NixOS config? And why are directions like this in the imperative form? I'd think that people working this closely to core Nix would be pushing declarative setup so there must be something I'm missing.

Another example is nixops 2.0.

[pxc]

I think this is one of the things that flakes aims to solve. I'm playing around with a new flake-based setup and I'm using flake-utils-plus[1], and I use the overlays builder to make it easy to refer to packages that come from flake inputs in my configuration.nix and similar places, like. I just added the latest Nixops from master to make sure I was giving you a real answer, and I use an overlay like this in my flake.nix:

      (final: prev: {
        nixops = inputs.nixops.defaultPackage.${prev.system};
      })

after adding the flake input for nixops:

      nixops.url = "github:NixOS/nixops";

to my system flake's `inputs` attribute. And after a `nixos-rebuild switch`, nixops reports the highly following mysterious version number :

   nixops --version
  NixOps @version@

If you're not using Nix flakes, you can do something similar with Nix and Nixpkg's fetchers as well, also using overlays, using the `nixpkgs.overlays` option in NixOS. This is how, for example, the community Emacs overlay recommends folks use it[2].

What you wanna do for projects that don't offer a flake.nix or a ready-made overlay is to use a fetcher like in the Emacs example, but write your own overlay function that invokes Nixpkgs' `callPackage` function on whatever Nix expression inside the repo represents the package you want, or imports them from a release or default.nix as appropriate. Home-manager uses this to define its own little overlay[3], the one that gets used in its flake.nix, and its package is just its default.nix[4].

Unfortunately, in the pre-flakes world, you just have to read the Nix code and figure out how to translate things to get the attributes you want into scope. For example, this works for nix-processmgmt:

      (final: prev: {
        nix-processmgmt-tools = (import (
          (builtins.fetchTarball "https://github.com/svanderburg/nix-processmgmt/archive/6def8584c6b028c922c550859a07b989d21d6f73.tar.gz")
            + "/tools/default.nix")
          { pkgs = prev.pkgs; }
        );
      })

and then you can add, e.g., `nix-processmgmt-tools.common` to your `environment.systemPackages`.

I think maybe the reason instructions aren't given for some projects is that they see those parts of the guide as mostly for beginners or casual experimenters, and they expect advanced or ‘serious’ users to be able to figure it out without much trouble. To some extent I think this is because different people choose to pin packages in the pre-flakes world through a variety of different mechanisms, and the authors of these packages and tools don't know in advance how you want to do it.

In the case of nix-processmgmt, I think Sander doesn't actually expect to have any users! In such cases, an imperative install that users are expected to play with for a little while and then just throw away is supposed to be enough.

You're right, though, that it's odd and disappointing that instructions for the preferred way of doing things are sometimes simply not given. A polite pull request or issue report would probably be well-received. My recommended strategy, if you get stuck, would be to ask for help getting those packages into scope declaratively on Discourse or Matrix, and then to offer the authors of these out-of-tree packages pull requests to modify their READMEs accordingly. :)

PS: You don't necessarily _have_ to use overlays for these. You can also drop expressions that use builtins.fetchTarball and then use `callPackage` or import from those sources directly into lists of packages like `environment.systemPackages`.

—

1: https://github.com/gytis-ivaskevicius/flake-utils-plus

2: https://github.com/nix-community/emacs-overlay#quickstart

3: https://github.com/nix-community/home-manager/blob/master/ov...

4: https://github.com/nix-community/home-manager/blob/master/de...

[mason55]

Wow, a lot to digest here, thanks for the super detailed response! Definitely going to be coming back to this over the next week or two and trying some things out.

> Unfortunately, in the pre-flakes world, you just have to read the Nix code and figure out how to translate things to get the attributes you want into scope

This has been my approach, I always kind of assumed that there must be a better way but at least I feel a little better knowing that there isn't really. I haven't spent much time with flakes because I mostly only use Nix as a package manager/operating system but this helped me see how even if I'm not personally writing packages that I want to re-use I can still get a lot of value from flakes, so getting that setup is probably my next step!

[takeda]

I don't think removing it is a good idea. Nix is a paradigm shift and things are done completely different, nix-env though is still one piece that might be somewhat similar to what people are used to.

[pxc]

I like to use nix-env (or more often, now, `nix profile`) for a persistence level in between nix-shell and really adding something to my home-manager or NixOS config. I let my profile build up 10-20 things installed, then every few weeks I decide what belongs in my declared config and uninstall everything in the profile.

Imo entirely removing imperative package management would be a mistake, although imperatively managing a file that gets sourced in your declarative config (a bit like /var/dpkg/selections on ol' Debian) instead of putting the whole profile manifest in the Nix store and leaving it at that would be better.

[mason55]

> I let my profile build up 10-20 things installed, then every few weeks I decide what belongs in my declared config and uninstall everything in the profile.

I'm curious what you get out of this that you don't get from just adding/removing packages in your home-manager config? Is it just a matter of it being quicker to do nix-env -iA instead of updating your config and running home-manager switch? Or is there some other benefit?

[lloeki]

I do have that use case as well. Taking a parallel from git, it's kind of a working/staging area/stash. There's a feeling of zero commitment. It could be I try out some unknown package, or that I have a temporary one-shot need that still lasts longer than a nix-shell -p.

But mostly I use it as a beachhead for people to eventually jump ship, helping them climb the first step of the ladder:

"see? It's easy to install and use, you'll be quite autonomous. And if you feel it's not your thing just ignore it or rm -rf /nix"

Then I bait them with e.g shell.nix just enough to tease their curiosity.

So instead of feeling overwhelmed by a whole new arch and contractually tied by configs they feel free, empowered, and curious.

[pxc]

> Is [the benefit of this approach] just a matter of it being quicker to do nix-env -iA instead of updating your config and running home-manager switch?

That's definitely a factor. I think since I keep my Nix configurations in source control, somehow modifying the configuration feels more ‘official’, and it also usually comes with extra steps like committing and pushing.

The other thing I like is that it makes it very easy to tell if I actually want/need something: if I find myself installing something over and over (because I periodically purge my profile), I know for sure that it's time to add it to my config. This way I end up pulling less crap I don't actually use into my setup in an enduring way.

Maybe I'd also feel the same way about invoking things via `nix run` or `nix-shell` over time, and that would motivate me to incorporate them into my config ‘for realsies’ by declaring them.

> instead of updating your config and running home-manager switch?

I'm not currently a home-manager user on NixOS. Before home-manager was a thing, I used to define groups of packages using buildEnv and store them in an overlay for very simple de-facto declarative package management, so something like:

  nix-env -iA pxc-tui-apps

would install my whole CLI environment at once, and then on NixOS, I'd include `pxc-tui-apps` in my `environment.systemPackages`.

I'm switching to home-manager in my new setup, but one thing I still like about `nix profile install ...` (the flakes-based/next-gen replacement for nix-env, currently) is that it's user-mode/unprivileged, and it doesn't involve rebuilding my whole system (or anything other than the dependencies of just the package I want), even if my nixpkgs checkout/channel/flake registry or whatever has changed. `home-manager rebuild switch` is also unprivileged and also doesn't involve updating my whole system, but unfortunately home-manager doesn't support flakes just yet. You can use it on flakes-based setups on NixOS and macOS via the home-manager NixOS and nix-darwin modules, respectively... but then you're giving up the other benefits I like, because you have to invoke nixos-rebuild after all!

If `nix profile` were some day removed along with `nix-env`, but I had a user-level declarative environment management tool (like home-manager or something more tightly integrated), I could probably get by with just a little discipline about how I choose to edit my configurations and manage sources of Nix expressions and be pretty happy.

But I think the problems with `nix-env`/`nix profile` are pretty solvable, and I think lacking any imperative solution at all will likely put some ‘winnable’ new users off.

I do agree that `nix-env` itself sucks and needs to go, though. `nix-env --upgrade` doesn't really do what people expect, and there's no real reason to use/prefer it over just removing/reinstalling a package. The way that `nix-env` thinks about versions is basically insane, since Nixpkgs doesn't have any real version metadata and `nix-env` just parses attribute names to get the versions back out. `nix-env --query` is clunky and slow (but the new `nix search` is awesome and crazy fast!). `nix-env` was a cool thing for its time, and it's actually how `home-manager` manages its profiles on the backend (which is why flake support is still lacking; enabling flakes disables `nix-env` for your user). But it's like an imitation of `rpm`, and it was made before Nix had a real userbase and opportunities to think about what operations/abstractions/metadata were desirable at that level.

One of the things that's cool about Nix's design is that its design allows you to just bypass the hardest and most annoying problem that faces traditional package managers: dependency resolution. If you want a package manager that's guaranteed to give you solutions that are correct and complete, you need a SAT solver for dependency resolution, and that's NP-complete. By leveraging its quasi-content-addressable derivation approach, Nix gets to avoid resolving dependencies like traditional package managers do— the thing you need is the thing you were built against, and that's that! Similarly, by leaning hard into the Nixpkgs monorepo so that almost all packages live on it and all third-party package sources are built as de facto overlays on top of it, Nix has been able to totally avoid having to think about versions.

Compared to formats like DEB or RPM, Nix packages have very, very little metadata. Nix packagers don't have to declare things like acceptable version ranges for dependencies, what packages provide, what package names ought to be considered equivalent, what other packages it's incompatible with, or even what version number a package has. But `nix-env`'s `-q`, -i`, and `-u` options all imitate the `rpm` CLI, where all of that kind of metadata is necessary and present. And `nix-env` makes all that stuff work by assuming the structure of Nixpkgs and operating on Nix attributes representing packages directly. And it doesn't really make sense in the Nix world as it exists.

Flakes is the first attempt to revisit all these questions the community has basically punted on like ‘how do we want to relate packages to each other in a way that's not monorepo-centric?’, ‘what kind of metadata do we actually want to have for publishable Nix source package artifacts?’, ‘how should Nix code repositories advertise what features/tooling (packages, shell environments, modules, overlays, whatever) they support?’, ‘do we want Nix to actually be able to reason about versions?’, etc. (This also possibly reintroduces the question of dependency resolution. Hopefully not?) I think once we have real, considered answers for questions of that kind, grounded in the experience of the community so far, we can build an imperative frontend to Nix that makes sense and is nice to use.

[lloeki]

> or even what version number a package has

In the context of within nix packages that's correct. In the context of nix usage that is not.

I start a project, I need it to use Ruby 2.7.x, OpenSSL 1.1, node 14.x because that's what the project is compatible with, and I need that to happen on both Darwin, Linux, on whatever cpu arch. Pinning the hashes of "whatever I built it with" won't work.

Worse, some software such as e.g Ruby encodes their platform at build time (because it matters, because #ifdefs) so currently I'm on darwin20 but specifying "ruby" pulls in RUBY_PLATFORM==darwin17. Nix is currently helpless in face of that.

> and it doesn't involve rebuilding my whole system

True. One thing I'm 100% sure is that nix-env -i will do just that and nothing else, whereas nixos-rebuild switch might include something else pending I might have forgotten about because it relies on globals.

[pxc]

> I start a project, I need it to use Ruby 2.7.x, OpenSSL 1.1, node 14.x because that's what the project is compatible with, and I need that to happen on both Darwin, Linux, on whatever cpu arch. Pinning the hashes of "whatever I built it with" won't work.

> Worse, some software such as e.g Ruby encodes their platform at build time (because it matters, because #ifdefs) so currently I'm on darwin20 but specifying "ruby" pulls in RUBY_PLATFORM==darwin17. Nix is currently helpless in face of that.

Right, right. That's a real problem. Gentoo Portage has mostly a big monorepo kinda like Nixpkgs, and in it you can find multiple versions of many pieces of software. Maybe the sensible future in the Nix world would be:

1. Our package attributes include version metadata, perhaps through something like ‘subflakes’ within the repo.

2. Nixpkgs includes multiple versions of major pieces of software.

3. Nixpkgs' top-level remains basically as it is now, in that for the most part only the latest version of something is used as a dependency. Alternatively, we use some kind of a ‘lock file’ that gets published with Nixpkgs. This way `nix profile install` still doesn't have to perform any dependency resolution for packages inside nixpkgs, so its behavior stays fast and predictable.

4. You can add version constraints in defining packages for use outside Nixpkgs, in `nix shell` environments, etc.

Does that seem like the way to go for you, or is your ideal picture something else?

[lloeki]

> If you want something "installed" use home-manager.

I hear you. I understand nix-env as it exists needs to go, and I can only trust you on the support side. I did not know about home-manager.

Doing my homework, from home-manager README, I can read:

> Before attempting to use Home Manager please read the warning below.

> Unfortunately, it is quite possible to get difficult to understand errors when working with Home Manager, such as infinite loops with no clear source reference. You should therefore be comfortable using the Nix language and the various tools in the Nix ecosystem.

> Home Manager targets NixOS unstable and NixOS version 20.09 (the current stable version), it may or may not work on other Linux distributions and NixOS versions.

> Also, the home-manager tool does not explicitly support rollbacks at the moment so if your home directory gets messed up you'll have to fix it yourself.

On top of being third party (for now?), all of this really does not bode confidence in the tool and severely raises the bar for adoption when all one wants is to install tmux globally for their user (IOW a bunch of symlinks in ~/.nix-profile/bin that "just works").

But the best one is this in the manual:

> This manual will eventually describes how to install, use, and extend Home Manager.

There is no section in the manual describing the usage of the tool. The Getting Started seems to be solely about development and contributing. The terse examples in the README is what made me search for the manual, and I could only get a grasp of what they entail because I have NixOS experience. Seriously, if this is deemed "general public availability" quality, this is borderline user hostile.

So, I do note the envisioned deprecated-ness of nix-env but I will continue to use that as a first rung to help people climb the ladder when I introduce people to nix until there is a suitably accessible and reliable replacement.

Indeed I successfully used it as a beachhead for people to eventually jump ship:

"see? It's easy to install and use, you'll be quite autonomous. And if you feel it's not your thing just ignore it or rm -rf /nix"

Then I bait them with e.g shell.nix (which feels like Gemfile) or nix-shell -p --run (which feels like docker run) just enough to make it relatable while teasing their curiosity.

So instead of feeling overwhelmed by a whole new arch complete with a foreign config system and an alien language, and contractually tied by configs they feel confident, uncommitted, free, empowered, and curious, and that much more likely to transition to the next rung up.

> If you just want something for quick dev use nix-shell.

That I 100% agree with, which is why I immediately described it as well. It's the hook to the next rung.

> In some cases Home Manager cannot detect whether it will overwrite a previous manual configuration.

That, to me, is a problem. I fully acknowledge the limitations and caveats of nix-env, and duly highlight them when introducing people to nix, but this means home-manager is not the tool I need, in git parlance it's way too much "porcelain" and not enough "plumbing". It feels invasive when you're still in the process of getting acquainted.

I do not want home-manager to handle launchd, nor .gitconfig. I do not use nix-darwin or NixOS, _on purpose_, as I use nixpkgs as an additional tool, an extension of existing systems.

I am fine if nix-env has some of its features revamped, or is deprecated and replaced by a better tool but in terms of some of its existing use cases it is exactly the tool needed, i.e merely allow users or root to make a package persistent and widely available for the user or for root in their PATH, as if it were a native package, and with no other side effects.

I hope you understand these are legitimate needs.