[k32]

Nix is a nice research project, but after playing with it for ~6 months and contributing to nix-packages, I came to conclusion that (in my case) it's not suitable for production nor development, and now I manage my personal machines with a simple Ansible playbook.

Main reason for leaving was that Nix package maintainers have to heavily patch all software. Packaging for nix is more like porting software to another operating system. Just check the amount of ad-hoc patches in nix-packages repo, and note that they have automated tools for patching the most common problems, so the problem is even worse. Now how about quality of these patches... I was able to merge some pretty large patches to nix-packages from an anonymous and sketchy-looking github account, and they weren't scrutinized much, because the original author of the derivations abandoned them.

Moreover, Nix breaks the chain of trust for the language packages. For example, Erlang and Elixir packages are signed cryptographically. Most erlang libraries come with a rebar.lock file containing hashes of the dependencies, so reproducible builds are already ensured. Unfortunately, package hashes are incompatible with Nix derivation SHA's, because they include some additional envelope data. What did Nix people do to work around this problem? They patched out the checks for rebar.lock files from the 3rd party Erlang build system. I would not dare to run a distro that contains patches like this in any kind of production environment.

Additionally, /nix directory is readable by all users, so you cannot use Nix to manage secrets, and there was no universally approved way of doing so (or at least it was the case at the time I was using it).

As for personal use... You cannot install opam packages without dealing with incompatiblities, you cannot easily install games from Gog without dealing with incompatibilities, and to be honest, do you really need such degree of reproducibility for your very own dev machine?

[pstch]

As said by the other commenter, most of these patches are caused by NixOS not using the FHS (except for /usr/bin/env), which can be considered as a good thing in itself. In an ideal world, it would not be a problem (programs should not assume anything about the filesystem hierarchy, and should always use $PATH or their own environment variables).

The issue of integrating with other language ecosystems is indeed very problematic (and it's a very hard problem), especially with regards to the chain of trust (by the way, it's not completely broken, you can still read the original package hash in the derivation, and evaluate this derivation to get the Nix derivation hash), and with regards to the fact that including all of the packages of every ecosystem in `nixpkgs` is not doable.

There is also the (related) problem that evaluating a configuration implies evaluating `nixpkgs` itself, which requires a good amount of available memory, and this can only get worse as `nixpkgs` grows.

[lilyball]

I don’t know about Erlang but typically language-specific packages are handled with a tool that converts from the language-specific package management system into Nix, and that tool will do whatever validity checking you normally expect from the language package manager. For example, for Ruby, it uses bundler under the hood which will use the Gemfile.lock, and then it converts the results of that into Nix expressions.

Someone could certainly submit a PR to the nixpkgs repo that purports to do that but really modifies the generated Nix expressions to refer to different sources, but this would be discovered by anyone who re-runs the package update process.

[nextos]

I maintain several Nix packages, and I agree that for some edge cases it can be a pain.

Another annoying thing is that NixPkgs has developed some technical debt, with some packages being perennially broken. Guix is smaller, but much cleaner in this regard. Julia can't be used in Nix, and many Bioconductor packages are broken.

However, I feel like the existence of FHSUserenvs needs to be better advertised and documented. These give you the ability to easily install unpatched software that assumes a regular (FHS) Unix filesystem layout (/bin, /lib...).

Still, I think the benefits of using Nix and NixOS surpass the drawbacks right now. I use Nix in dozens of machines. It is, along with Arch, my favorite distribution. Both sit at two local maxima of the design space. Declarative-functional and imperative with a minimal layer of tooling.

[bamboozled]

If you had to go with one or the other, which would it be? I'm on the fence now, I have one machine with Arch, my heavy development machine with NixOS. I'm considering ditching NixOS altogether.

I just miss using the same tooling as everyone else in my team without issues or having to constnatly spawn up a new nixenv everytime I change folders. Maybe using FHSuserenvs will help?

I've never really been able to grasp the language, I don't care what anyone says, the documentation is terrible. Every time I try build a package it always takes me days to workout how to do it.

Really feeling that while, yeah, it's nice to have everything in one configuration file, it's good, but when something breaks, I have really no idea how to fix it.

[setheron]

To be fair, the packages have to be patched because many applications assume FHS.

The fact that Nix ditches FHS is so what's appealing about it unfortunately there's a huge body of work that assumes it.

Most of the patches though are making libraries or binaries fully referenced from nix store rather than from the PATH or some other implicit state.

[kris-s]

What is FHS?

[pstch]

> The Filesystem Hierarchy Standard (FHS) defines the directory structure and directory contents in Linux distributions.[1] It is maintained by the Linux Foundation.

https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard

[otabdeveloper4]

> Nix is a nice research project

It's not. As far as I can tell, it's used heavily in some corporate environments once you're faced with solving the development environment dependency hell and you need something that actually solves your problem. (This is my story too.)

> ...and now I manage my personal machines with a simple Ansible playbook

Where Nix is used you typically don't have "personal machines". And having a separate guy just doing setups of developer environments is a huge overhead.

[foolmeonce]

I have traditionally used ansible and I hate it and am slowly converting to nix tooling.

With ansible, I feel like everything is poorly solved and you need to weigh decisions about when to use various hashicorp products. Worse still if you use their plugins instead of just script substitutions you get stupid behavior like not being able to fix your configuration because it is broken, or turning off the supposed advantage of these plugins such that they are almost as good as sed again.

[bak3y]

I've used ansible for years to configure my local workstation and hundreds of linux servers both on prem and in AWS and have never needed a Hashicorp product.

[Ericson2314]

I don't find myself patching stuff all the time, and would hate to go back to half-assed leaky-abstraction kludges like Ansible.

But I do agree we need to deal with the language-based packaging situation much better. It's a long-standing project of mine to fix that.

[k32]

> I don't find myself patching stuff all the time, and would hate to go back to half-assed leaky-abstraction kludges like Ansible.

Speaking of half-assed kludges, NixOS itself is a held together by duct tape of scripts patching shebangs and whatnot.

[Ericson2314]

Those are kludges, but they are nicely confined to build time.

The deployed system is very easy to reason about, which I greatly appreciate with my laptop and production environments.

(In addition to trying to support the language-based ecosystems better, I also want to reduce the things we do in bash. I think those are complimentary goals, too.)

[carapace]

> They patched out the checks for rebar.lock files from the 3rd party Erlang build system.

Oh ffs really? Yeah, that's the kind of hijinks that leave you scratching your head. Thanks for the head's up.

[soraminazuki]

No, the commenter is grossly misrepresenting the facts here, as is the case with every other point being made in that same comment. The patch in question disables the ability to fetch packages from online sources, requiring you to specify the URL of the required packages and its hash beforehand. Plus, it's not even meant for general use and instead is meant to help with packaging for Nix. For general use, you can very easily use the unmodified build instead.

[carapace]

Cheers

I'll admit (although it annoys me) that I couldn't follow what was going on there. Nix x Erlang's gnarly build system seems like complexity junkie crack to me, but then I'm getting older.

[soraminazuki]

I too initially assumed the commenter was being honest, and it took me a while going through code, commit history, and GitHub issues to realize that this wasn't the case.

[k32]

> Erlang's gnarly build system

It's actually fine, thanks for a remark.

[k32]

> requiring you to specify the URL of the required packages and its hash beforehand.

...a different hash.

You see, I naively think that a checksums should not be overwritten when they doesn't match.

[soraminazuki]

That's not what I said. It's not about replacing hashes.

[k32]

> Oh ffs really?

Yes, really. This PR partially reverted this behavior (or rather made it less intrusive): https://github.com/NixOS/nixpkgs/pull/54115/files , but from there you can easily track "rebar3 hermecity patch", which had introduced it.

[carapace]

Egad!

[StreamBright]

You just saved me a ton of time. I wanted to investigate if I should try Nix but this is not going to fly. Thanks for the detailed comment.

[soraminazuki]

Almost every point being made in that comment grossly misrepresents the facts and/or leaves out key details. For specifics see this comment[1].

If you're curious about Nix, I highly recommend you give it a try. nixpkgs have great quantities of packages that matches Arch/AUR, and every single one of them goes through mandatory reviews ensuring quality. On top of that, the Nix tooling makes it easy to create your own packages and reuse existing package definitions. So for example, if you want to use a fork of a software available in Nix and maybe even add some custom patches, all you need to do is write a few lines in your config. Switching to Nix was a really refreshing experience and the amount of documentation and the code in the nixpkgs repository was very helpful for me to get started.

[1]: https://news.ycombinator.com/item?id=25030546

[Ericson2314]

Plenty of us do use it on our own machines and in production, and it doesn't seem like a never-ending packaging war.

Most software actually doesn't assume the FHS. (Autoconf, Meson, and CMake all don't, for example.)

[soraminazuki]

> Main reason for leaving was that Nix package maintainers have to heavily patch all software.

Any widely used distros do this. Plus, the statement that you have to patch all packages is false. In many cases, all you have to do is set the right build flags and environment variables, which is handled by standard helpers in nixpkgs.

I'd also like to add that nixpkgs has the most extensive testing for packages compared to any other distros I've seen.

> Packaging for nix is more like porting software to another operating system.

Dramatic, but again, simply not the case.

> note that they have automated tools for patching the most common problems, so the problem is even worse.

This is a good thing. Manual patching for the most common problems isn't sustainable, especially if you maintain a large number of packages. Just ask packages maintainers of any major distros.

> I was able to merge some pretty large patches to nix-packages from an anonymous and sketchy-looking github account, and they weren't scrutinized much

All packages in nixpkgs has a mandatory review process as you should very well know. As scary as you make it sound, I'm pretty sure at least one committer have taken a look at your code to see if there was anything malicious in it.

> Moreover, Nix breaks the chain of trust for the language packages.

After a quick look, it seems like you can still use the unmodified erlang package manager? The patched version you've mentioned seems to be there for facilitating creation of Nix packages and it looks like the main intent was to remove the ability to download packages from the internet. If this is the case, it wouldn't be a security problem at all because the dependencies would have to be prefetched and their hashes precomputed.

> /nix directory is readable by all users, so you cannot use Nix to manage secrets

Why not store secrets in a private location, just like you would on any other distros??? What is it that other distros have that Nix is missing? I'm not seeing your point here.

With that being said, note that this feature is being worked on, and systemd very recently introduced a new feature for secrets management [1].

> You cannot install opam packages without dealing with incompatiblities, you cannot easily install games from Gog without dealing with incompatibilities

Matbe take a look at buildFHSEnv.

> do you really need such degree of reproducibility for your very own dev machine?

Yes, it saves me from having to do a clean install of my system every few months. I haven't done a clean install at all the last few years since I started using NixOS.

[1]: https://github.com/NixOS/rfcs/pull/59#issuecomment-624723733

[k32]

> Any widely used distros do this.

I've yet to encounter another distro that deliberately breaks 3rd party software the way Nix did it to rebar3 (before it was reverted).

> Plus, the statement that you have to patch all packages is false.

IIRC `patchShebangs' script runs for any package using stdenv.

> I'd also like to add that nixpkgs has the most extensive testing for packages compared to any other distros I've seen.

Which distros? Can you share a comparison in number of tests/coverage?

> Dramatic, but again, simply not the case.

As other commenters mentioned, software written for GNU\Linux often assumes FHS. NixOS breaks FHS, and has to patch soft to work around that.

> This is a good thing.

How so?

> This is a good thing. Manual patching for the most common problems isn't sustainable, especially if you maintain a large number of packages. Just ask packages maintainers of any major distros.

Other major distros don't break FHS, therefore they don't need to patch trivial things like shebangs.

> All packages in nixpkgs has a mandatory review process as you should very well know. As scary as you make it sound, I'm pretty sure at least one committer have taken a look at your code to see if there was anything malicious in it.

It depends on your threat model. Mine is stricter.

> If this is the case, it wouldn't be a security problem at all because the dependencies would have to be prefetched and their hashes precomputed.

Once again, it breaks the cryptographic chain of custody set by the upstream. In my threat model that's not acceptable. I tend to trust the upstream more than the distro.

> Why not store secrets in a private location, just like you would on any other distros???

Secret provisioning is by far the most difficult DevOps problem, and it can go catastrophically wrong for all too many reasons. Nix not only doesn't help, but also introduces additional footguns with world-readable configuration files.

> What is it that other distros have that Nix is missing?

Other distros have FHS. FHS accounts for secret management, among other things.

> Yes, it saves me from having to do a clean install of my system every few months.

Ok, though I never had to do a clean install of GNU\Linux either on my dev machine.

[aidenn0]

The fact that they reverted the rebar change suggests that or wasn't intentional.

[soraminazuki]

> I've yet to encounter another distro that deliberately breaks 3rd party software the way Nix did it to rebar3

Except that your anecdote is completely misleading given that the patched version was intended for creating Nix packages. You could've just used the unmodified version instead.

Furthermore, upstream developers complain about how distros are breaking their software all[1] the[2] time[3]. Again not a Nix specific problem. Nix even alleviates some of the problems because you can use different versions of the same software, meaning that different packages don't have to share dependencies as aggressively as other distros do.

> Which distros? Can you share a comparison in number of tests/coverage?

Arch, Debian, CentOS, and Homebrew, though the last one isn't strictly a Linux distro. I can't possibly count the numbers given that the sheer number of packages available, so it's based on experiences at looking at package definitions. I tend to check package definitions across distros a lot to write packages fir Nix.

> It depends on your threat model. Mine is stricter.

If you have a problem with software projects accepting outside contributions even if they go through review, you have a problem with using free and open source software in general. It makes me wonder how you even manage to use the Linux kernel.

> Once again, it breaks the cryptographic chain of custody set by the upstream. In my threat model that's not acceptable.

Then use the unmodified rebar3 for your project instead. But anyways, if your threat model involves not trusting SHA256, you likely have a problem with your said cryptographic chain of custody too.

> Nix not only doesn't help, but also introduces additional footguns with world-readable configuration files.

> FHS accounts for secret management, among other things.

That's an outright lie. NixOS doesn't prevent you from storing secrets in a file and applying your usual UNIX permissions. FHS has absolutely nothing to do with it.

[1]: https://github.com/pypa/pip/issues/5599 [2]: https://web.archive.org/web/20181021091049/https://github.co... [3]: https://twitter.com/videolan/status/1153963312981389312

[k32]

> Except that your anecdote is completely misleading given that the patched version was intended for creating Nix packages. You could've just used the unmodified version instead.

The package was called rebar3, not rebar3-nix or something. If the fork was called differently then no one would complain. Also this fork was based on hacks, like patching rebar3's internal files and messing with caches. If you don't see anything wrong about this, then this way of thinking just proves what I said in the parent.

But forking rebar3 wasn't needed at all, because rebar3 _already_ had all the features needed for Nix. Namely, it has get-deps command producing fixed output derivation, and it supports _checkouts directory, needed for hermetic build.

> Then use the unmodified rebar3 for your project instead. But anyways, if your threat model involves not trusting SHA256, you likely have a problem with your said cryptographic chain of custody too.

There's no reason to use SHA256, if you just replace it with a different SHA256 whenever it doesn't match with the upstream.

> Nix even alleviates some of the problems because you can use different versions of the same software, meaning that different packages don't have to share dependencies as aggressively as other distros do.

Docker exists. Although it's not perfect by any means, it doesn't require reinventing the entire world. I extrapolated my experience with Erlang to the rest of NixOS reinventions, which in theory could be wrong, but it was convincing enough for me.

> That's an outright lie. NixOS doesn't prevent you from storing secrets in a file and applying your usual UNIX permissions. FHS has absolutely nothing to do with it.

I was not talking about "storing secrets in a file", but rather about the whole live cycle of the secrets, including transferring them between parties, storing, provisioning, rotation, etc (imagine you have to solve a problem of changing some secret on 100 machines securely and reproducibly via blue/green deployment). As far as I know, Nix doesn't provide facilities for solving this problem at scale. I would be happy to be proven wrong.

> FHS has absolutely nothing to do with it.

It absolutely does. Some pieces of software look for secrets in /etc/ (Kafka, for example). When etc (or its analogue in /nix) is world-readable hilarity ensues.

[soraminazuki]

> The package was called rebar3, not rebar3-nix or something. If the fork was called differently then no one would complain.

First of all, the fork obviously isn't there for general consumption. You made it sound like unsuspecting users would use it and have unverified code downloaded on their machines, which is borderline false when it doesn't have the ability to download code from online sources. This attempt to mislead is what I'm having problems with.

Second, it would've been nice if you've raised an issue on GitHub and helped improve Nix instead of complaining and spreading borderline falsehoods on HN. Erlang isn't exactly a widely used language, so design choices would be made by the few who actually uses it. It would always help to have more productive input by actual users.

> Also this fork was based on hacks, like patching rebar3's internal files and messing with caches. If you don't see anything wrong about this, then this way of thinking just proves what I said in the parent.

Ranting about how "it's a hack!" based on cherry-picked implementation details isn't convincing at all. The intent here is to have a copy of rebar3 that builds packages from local sources only, for consumption by package maintainers. That's not by any means unreasonable.

> But forking rebar3 wasn't needed at all, because rebar3 _already_ had all the features needed for Nix.

Tell me a single major distro where it's okay to essentially redistribute packages from language-specific package managers without making huge adjustments in the way packaging is handled. I know of none. For countless software, distros just give up on packaging altogether because the effort required to make that adjustment can easily get out of hand. This is the major reason many Maven packages aren't packaged on Debian.

To make matters worse, many packages from languages-specific package managers won't even compile on major traditional FHS compliant distros. Changes to upstream packaging aren't unwarranted, however much you'd prefer to believe otherwise.

> There's no reason to use SHA256, if you just replace it with a different SHA256 whenever it doesn't match with the upstream.

The SHA256 would be verified by package maintainers, reviewers, and the hydra CI instance.

> Docker exists.

If you're against the concept of distro-provided packages, why didn't you say so upfront instead of making baseless accusations about what a horrible security disaster Nix is?

Also let me remind you that almost all Docker container images rely heavily on distro-provided packages. What are you going to do now?

> As far as I know, Nix doesn't provide facilities for solving this problem at scale.

Neither do other distros, though Nix is a close to having a built-in solution as I've just shown you.

> When etc (or its analogue in /nix) is world-readable hilarity ensues.

Why are you repeating this when I just told you it's not true? /etc in Nix is not world-readable, plain and simple. This works perfectly fine on my NixOS system:

    $ echo secret | sudo tee -a /etc/secret > /dev/null && sudo chmod 600 /etc/secret    
    $ cat /etc/secret
    cat: /etc/secret: Permission denied
    $ sudo cat /etc/secret
    secret

[k32]

> First of all, the fork obviously isn't there for general consumption.

I didn't know that when I installed a package called "rebar3" and spend some time trying to understand what's wrong with my build. Oh, my bad, should've known better that "rebar3" was for internal consumption, and I need to install "rebar3-open" instead.

> Second, it would've been nice if you've raised an issue on GitHub and helped improve Nix instead of complaining and spreading borderline falsehoods on HN.

Thanks for suggestion. I did exactly that, and I fixed this entire mess. In fact, the current NixOS Erlang infra is partially based on that work.

> Ranting about how "it's a hack!" based on cherry-picked implementation details isn't convincing at all.

I professionally work with Erlang, and I know what is hack and what is not.

> Tell me a single major distro where it's okay to essentially redistribute packages from language-specific package managers without making huge adjustments in the way packaging is handled. I know of none. For countless software, distros just give up on packaging altogether because the effort required to make that adjustment can easily get out of hand.

Language-specific packaging should be left to language-specific package managers, they have more domain knowledge. Nix doesn't solve this dilemma in any better way.

> Also let me remind you that almost all Docker container images rely heavily on distro-provided packages. What are you going to do now?

Keep on using it? Take systems software from the distro-provided packages, add my own stuff on top. Building RPM is a nobrainer. I was considering Nix as an alternative to our deploy process, but realized that it would introduce more problems than solve.

> /etc in Nix is not world-readable, plain and simple.

Please take a closer look at what you're quoting. You missed a very important part:

> (or its analogue in /nix)

You see, if I want to utilize Nix to full extend and use its main "reproducibility" feature, I need to put all my configuration in a derivation. Configuration includes secrets. Derivations are world-readable.

> Neither do other distros

Distros maybe don't, but "bug-ridden kludge that is Ansible" (according to Nix user) does.

[aszlig]

Agreed, the language support situation is a very mixed one in nixpkgs, where support ranges from rudimentary (or even non-existing) to well integrated and supported. Particularly the Erlang/Elixir situation is - as you already brought up a while ago[1] - very rudimentary and really needs to be improved.

Right now, I do manage my Erlang dependencies with Nix only and without using rebar/mix (particularly since I do want full release upgrades with hot code reloading), but manually managing those dependencies certainly doesn't scale well for all projects, especially if they already use rebar/mix.

I also haven't looked into newer projects like nix-elixir[2], but as another commenter here mentioned, there are typically tools available to convert between the language-specific package managers, but they all come with their own set of issues and tradeoffs (eg. duplicating dependency information or importing from other derivations at evaluation time) so they might not be suitable for your project.

The situation with secrets is a long lasting issue[3] with several attempts of solving, but so far the best practice is to just to either leave secrets out of the Nix store or just make sure the entire Nix store is not accessible.

Since I also use NixOS for a few days, I can imagine why it could be frustrating if things don't work out of the box like on other distros. Having had my own adventures patching things like no tomorrow, I know sometimes the effort to do this could be quite time consuming, especially the GOG games[4] you mention.

So I'd say if you frequently use software that is problematic on NixOS and don't want to go through the hoops of patching or making workarounds like eg. Docker or FHS user environments[5], I'd probably stay off NixOS.

Personally however I think it is worth going through a few hoops, since to me it shifts the frustration of "my system broke during an update" to "just want to get X to work NOW" and among other nice benefits (eg. bisecting whole systems, rollbacks and/or using specific packages before they broke) I'd rather prefer a working and reproducible system.

[1]: https://github.com/NixOS/nixpkgs/issues/53834

[2]: https://github.com/hauleth/nix-elixir

[3]: https://github.com/NixOS/nix/issues/8

[4]: https://github.com/openlab-aux/vuizvui/tree/0aa8064087b895d2...

[5]: https://nixos.org/manual/nixpkgs/stable/#sec-fhs-environment...

[k32]

Thanks for a constructive comment.