[Hnus]

I have never felt more dumb than when trying to use and learn anything nixos related. Its awesome technology not meant to be used by humans. I am using cs.github.com, open ai chat bot and usually like 4 tabs describing the same thing from different months using different commands + few versions of what seems like official manual while still having zero idea what I am doing. It feels more like I am reverse engineering something rather than using it. There is something seriously wrong with what is acceptable ergonomics by whoever is designing the user interfaces. I really wish they fix all their problems because when it all works its awesome.

I wanted to give guix a shot does anybody know if its any better ignoring their stance on non-free software as I think you can work around it.

[grumbel]

Guix interface is a bit more friendly by default, but once you enable the 'nix' command on NixOS, there really isn't much of difference in terms of basic CLI experience.

The big different is the language and I think Nix wins here by a mile. Doing everything with Scheme just leads to layers of macro spaghetti that I really did not enjoy to dig through. Error messages that tell you absolutely nothing about what went wrong were pretty common. Nix has those too, but less frequently. Also with Nix you just use regular shell scripts snippets for the building the packages, Guix wants you to do it all in Scheme. Package selection on NixOS is much bigger.

Another big thing, Nix has Flakes, which make it trivially to turn all your Git repositories into Nix packages. Your Git repository becomes essentially a first class citizen in the package manager, making it completely trivial to run different versions of the same software. Guix has none of that, they still treat packages as a separate thing from the software itself and trying to add third party packages involves quite a bit of overhead. Easily up or downgrading individual software isn't possible as far as I can tell, you have have to roll back the complete Guix system to do so.

Basically, after switching from Guix to NixOS, I can't say I missed anything. NixOS just felt like a more polished and feature rich version of what Guix was doing, which given that Guix is basically a NIH version of Nix, is understandable.

[jyrkesh]

> Another big thing, Nix has Flakes, which make it trivially to turn all your Git repositories into Nix packages.

I have no doubt that this is true, but I still feel dumb about it. I run a Plex server (among a few other workloads) on a NixOS box, and just figuring out how to update the Plex binaries off of the official master branch was a multi-hour back-and-forth endeavor between StackOverflow suggestions and compiler errors.

I really do love Nix though. And once I got it working I felt a LITTLE bit smarter, AND it's worked flawlessly for months.

[benreesman]

I’ve said this elsewhere in the thread and might do so again but it bears repeating: use flakes. At first they seem like this weird semi-supported thing, but eventually you realize that Nix is badly broken without them.

Flakes aren’t “experimental”. “Experimental” is what happens to your sanity without them.

[yjftsjthsd-h]

> Flakes aren’t “experimental”. “Experimental” is what happens to your sanity without them.

Okay, I like flakes, I use flakes, I enable flakes without much worry. But. They literally are an experimental feature, and they are still enabled by setting "experimental-features = nix-command flakes" in nix's config.

[benreesman]

I think I was maybe too casual with my language. Flakes are clearly marked experimental in the documentation. But they were a no-brainer a year ago, they’re always “a few months” from being official, they map directly onto users’ intuition around lock files, they enable principled composition of shit that doesn’t live in nixpkgs, the list goes on.

Outside of some (hypothetical to me) edge cases, not using them is masochism.

Flakes have a standard library (“flake-utils”) and and a pimped out version (“flake-utils-plus”).

Making it a weird gymnastics trick for a novice user to even access them is the kind of shit GHC devs say, “no, that’s too user-hostile even for us”.

Use flakes.

[ethbr0]

> a multi-hour back-and-forth endeavor between StackOverflow suggestions and compiler errors

Sounds like the authentic 90s linux package experience! (except with 's/StackOverflow/mailing list/')

[rowanG077]

How do you do it? I literally remove the flake from my nix profile and re-add it with a specific commit hash. I had troubles that just specifying master didn't update to the current master.

[jyrkesh]

I cheated and did a stateful thing (because I couldn't figure out flakes): sudo nix-channel --add https://nixos.org/channels/nixos-unstable nixos-unstable

and then I added this blurb to my Nix config:

nixpkgs.config = { allowUnfree = true; packageOverrides = pkgs: { unstable = import <nixos-unstable> { config = config.nixpkgs.config; }; }; };

[josteink]

> Doing everything with Scheme just leads to layers of macro spaghetti that I really did not enjoy to dig through.

On the flip side, it’s a real language, well understood, with a real specification, working tooling, and community support outside Guix (or in the case of Nix, outside the package manager).

As someone who gave up NixOS exactly because of the incomprehensibleness of the Nix language, I know what I would choose.

There’s no doubt at all.

[grumbel]

I see nothing especially incomprehensible about Nix, the language itself pretty simple. About the only thing I had a bit of an issue with is that I kept forgetting that functions only take a single argument in Nix. But other than that it is very nice to work with and has all the syntactic sugar you want, string interpolation, sets/map, sane multi-line strings and all that. Syntactic sugar is an area where Scheme has basically nothing to offer, everything needs calls to functions with long names.

For most common uses you barely even have to care about the language, as it's just some JSON-like data with shell scripts in between that do the actual work.

[parminya]

The language very much generates complaints because it is gives you no real help understanding what argument should be passed to what function and how to get that argument. Error messages are dreadful because there's often a million functions your object has to go through before it produces an error. Dynamic typing and lazy evaluation don't seem to mix. Some kind of compile-time, presumably gradual, type system would really be an incredible improvement, so that type errors were generally eager.

[pxc]

> Dynamic typing and lazy evaluation don't seem to mix.

Incidentally, this is one of the things that some of the people involved with the blog post in the OP have set out to fix. A few years ago, Théophane (author of the blog post) started an experiment to add gradual typing to Nix, which eventually became the start of Nickel: https://github.com/tweag/nickel

Today, Nickel development continues at Tweag, but now they have someone else, Yann Hamdaoui, driving the implementation forward. (I assume because he's a domain expert in programming languages.) Producing helpful error messages is one of the major goals of Nickel, as a potential successor language to Nix. All of the language design decisions are documented as GitHub issues on the repo, so if the technical side of this kind of problem is interesting to you, you may enjoy browsing that.

It appears that Nickel is very close to a usable state for early experimentation. You might be interested in cloning the repo and messing around with it!

[baby]

Honestly I’ve done enough scheme to understand that I wouldn’t touch anything related with scheme in a million years. People who think otherwise probably don’t understand why people like javascript and python and will never write any product that catches on.

The nix language is pretty nice on the other hand. The only thing I’m missing is a language server protocol thing so I can go to definition

[axlprose]

> People who think otherwise probably don’t understand why people like javascript and python and will never write any product that catches on.

Ironically, this has been posted on a popular site written in a dialect of scheme:

http://arclanguage.org/

https://github.com/arclanguage/anarki

[meltedcapacitor]

Where is the irony? HN does not expose scheme to the user.

Don't think the parent was saying that it's impossible to write a good program in scheme/lisp, more that scheme as user interface for a software system is a hard sell... which seems anecdotally true.

[baby]

how is this ironic? HN was written by one person, and nobody needs to care what language they used to make it work.

[t0astbread]

rnix-lsp is a language server for Nix that provides go to definition. I'm using it with the "Nix IDE" extension in VS Code. It doesn't handle imports though afaict, so it's only useful for local definitions.

[_hl_]

> Error messages that tell you absolutely nothing about what went wrong were pretty common.

Maybe I'm holding it wrong, but I almost never get useful error messages from Nix (NixOS and NixOps). Almost every error is deep within some module with no trace to the option I set. Just nix barfing its 300-line eval traceback where my own code doesn't even appear.

[geddawm]

Nix has a debugger now. You can drop into the frames that are causing problems and inspect the state. Miles easier than looking at the trace manually.

[benreesman]

I last flashed my flake.lock in June, so I might be out of date, but my rigs can’t reliably include the line number in my code that led to the duck-typing fiasco deep in the core of someLanguage.withPackages.fuck.WTF.this.

Fix the fucking “—show-trace” thing folks. Seriously.

[VTimofeenko]

Out of all cases when I used --show-trace, I can remember it being useful only once or twice. I gave up and just sprinkled builtins.trace.

IIRC debugger comes with nix 2.9 and nixpkgs for 22.05 still has 2.8.x.

[n8henrie]

I was excited to see the debugger flag to nix build the other day, since I'm really struggling cross-compiling an RPi on BTRFS root sd image.

Of course running with the debugger flag resulted in an unintelligible error message and no debugger.

[rtpg]

Any link to documentation on how to use it? Would be a godsend for me

[baby]

I agree. I'm actually trying a hello-world example from the tweag website, and it complains about "gcc" not existing. I try adding

    buildInputs = [ nixpkgs.gcc ];

but it complains that nixpkgs.gcc does not exist. There's nothing helpful. I remember fixing this issue a month ago, and I have no recollection on how I did it.

[hypersoar]

Are you using flakes? You need to use the legacyPackages output for your system, e.g.

  buildInputs = [ nixpkgs.legacyPackages."x64_64-linux".gcc ];

In practice, it usually looks something like this:

  let
    system = "x86_64-linux";
    pkgs = nixpkgs.legacyPackages.${system};
  in stdenv.mkDerivation {
    ...
    buildInputs = [ pkgs.gcc ];
    ...
  }

[benreesman]

So there is more than one way to skin this cat. ‘nixpkgs’ as handed to you as part of a flakes ‘inputs’ is (jargon alert) partially applied, which is to say you have to put a battery in it. In this instance what you have to pass is ‘system’ because nixpkgs supports many systems (e.g. ‘linux-x86_64’ or ‘darwin-aarach64’). It sort of makes sense that it wouldn’t know what GCC you want without that information.

You can do this up top by hand, e.g. P = inputs.nixpkgs { inherit system; }, but more commonly you would use a library like ‘flake-utils’ or ‘flake-utils-plus’ to issue system configuration for all supported packages to all your packages/derivations/devShells.

[baby]

Ah right, I saw that library but it didn’t have examples and I didn’t manage to use it. Do you know a page that would explain that?

[benreesman]

The ‘flake-utils’ readme is a pretty good jumping off point: https://github.com/numtide/flake-utils

I have this or that nitpick with FL and FLP but overall it’s very solid stuff. FLP is a little more “magical”, and that’s not always the best starting out, but you really can’t go wrong with either.

[Smaug123]

Maybe you got confused (like I did every time until I stopped doing anything non-flakey) about when `nixpkgs` is called `nixpkgs` and when it's called `pkgs`? In the flakes world it's considerably easier because everything is explicitly named, so you take an actual decision about what to call it.

[baby]

Is the convention to call it pkgs when you instantiate it with a system?

[cmm]

yes. other possible arguments are overlays and package selection modifiers (like whether to include non-free stuff or stuff that is marked as broken).

[kaba0]

I don’t know what is the specific file you are working on, but shouldn’t it be pkgs.gcc?

[SuperSandro2000]

That's strange. GCC is part of the stdenv on Linux and though always available.

To correct your coffee snippet: nativeBuildInputs = [ pkgs.gcc ];

[baby]

I’m on mac, I managed to fix it with https://mimoo.github.io/nixbyexample/flakes-packaging.html

[baby]

And now I’m trying to understand how devshells work in flakes and there’s pretty no doc

[benreesman]

So a dev shell as consumed by a flake is typically just a derivation. And typically the most important thing in that derivation is ‘buildInputs’, which is just a list of ‘pkgs.thingIWant’. It’s not uncommon to hook other “phases” of the derivation, which are (oversimplified) just little shell scripts that run in some order (‘configurePhase’ before ‘installPhase’) and typically expressed using the nifty “” syntax. For example if Nix is like stuffing paths into a Makefile that you want to use in your dev shell, you might put that in installPhase. Protip: a Nix “package” (derivation) evaluated in a string context (foo = “${pkgs.gcc}”;) gives you a path to where the thing lives in /nix/store.

[Smaug123]

Ask your questions on discourse.nixos.org please - the documentation situation in Nix-land is generally exactly as you have noted, which means centralised question-asking in well-known locations is much much more helpful for Those Who Come After.

[benreesman]

I say this with the utmost respect for those who work hard to answer questions on discourse, but I have found it singularly unhelpful.

I strongly encourage the Nix community to embrace a mainstream Q&A platform like StackOverflow or GitHub Issues.

Nix is “weird” enough for beginners already, the kind of janky interface and uneven moderation/quality on discourse is IMHO self-defeating for Nix.

[rekado]

Guix has package transformations, which lets you use different commits or git URIs for a given package definition.

It also doesn't require the definition of a package if all you want is an environment to hack on the code in a git repo. People use `manifest.scm` or `guix.scm` for these purposes.

Guix System services are in fact all built together, but upgrading individual software or services does not require "roll[ing] back the complete Guix system ".

(I want to point out that describing Guix as "a NIH version of Nix" is a symptom of the attitudes that made me almost completely retreat from discussions of software online. It's so grating to deal with dismissive attitudes like that again and again. Sucks all the fun out of hacking.)

[f1refly]

> Also with Nix you just use regular shell scripts snippets for the building the packages, Guix wants you to do it all in Scheme

I'm by no means a good schemer, but I'd still take scheme over shell every time. Shell tends to devolve into something completely unreadable with even moderate complexity while scheme can make even the most complex logic nice to work with.

> Guix has none of that, they still treat packages as a separate thing from the software itself

I'm not quite sure what you mean with that, but it's sure possible (and easy) to just define a guix repo that points to all your projects repositories and make them available through the guix package manager. Guix repos are just git repos with a few lines of metadata attached.

> Easily up or downgrading individual software isn't possible as far as I can tell, you have have to roll back the complete Guix system to do so.

It's very easy actually. You declare a package definition as an inferior to a package and can pin that definition to whatever version, commit hash, tag, you want. You basically say "I want the package declared as name X, but I want it while the guix repo is at point Y. That way, the package manager knows what dependencies the package has and provide those with the right versions as well.

> NixOS just felt like a more polished and feature rich version of what Guix was doing, which given that Guix is basically a NIH version of Nix, is understandable.

On the contrary, guix is more like a more polished and dedicated version of nix. Nix's solution to hard-to-package packages is "We take everything that's needed to build this thing and freeze it in the repository" while guix will not accept anything that doesn't build completly from scratch and can thus be distributed as optional source code. The difference is very important because it allows for better packaging and more flexible versioning of the packages. Using guix, you will never run into a situation where the solution has you staring down at a binary blob and questioning your life choices.

[Macha]

Is there an actual tutorial on how to use the nix command? All the official docs for Nix seem to be slightly discouraging it as not ready where they do mention it.

It's one where the community is pretty clear don't use nix-env etc. but the tutorials only explain nix-env, with the only details of the new commands being the CLI reference manual which doesn't cover workflows.

[phaer]

Yes, my understanding is that the official documentation lags behind community usage, because flakes are still an "experimental" feature as details of the implementation are worked out.

For reference documentation, there's "experimental commands" in the manual https://nixos.org/manual/nix/stable/command-ref/experimental...

[barkingcat]

I'm finding nix-env to be great! I do not understand why nix offers a command and a package index, but then advises not to use it.

If people aren't supposed to use nix-env, why don't nix put a huge deprecated notice on every invocation and delete the nix-env command itself from nix?

[Macha]

I think the problems with nix-env come down to two things:

- nix-env -i and nix-env -u will almost certainly eventually do the wrong thing for you if you use them enough (nix-env -iA is ok): https://ianthehenry.com/posts/how-to-learn-nix/ambiguous-pac...

- It's not reproducible as it depends on the state of your nix-channel setup at that time. How much of a problem this is depends on your use case. nix-as-homebrew-replacemnt or nix-as-shared-package-list-across-osx-and-linux users are probably fine with this, as what these users care about is more portability than reproducibility, but nix-as-predictable-dev-environment types will be let down by it.

I think the fact that the latter affects different users differently and that the new flake world doesn't offer a "modify your global environment without the tool taking over" option (expecting you to use nixOS or nix-darwin instead) has meant that there's not enough motivation to agree on a replacement that nix-env could tell you to use instead.

I guess the eventual replacement will be "nix profile", but while you'll find at least some documentation on "nix develop" and "nix flake", I had to look in the CLI tool docs for this one, which makes me suspect it's even less final than the other two

So instead the community will tell you to avoid nix-env, but there's no agreement to have the tool or docs tell you that.

[Smaug123]

It's not quite true that it's trivial to turn a Git repo into a flake, because flakes don't do submodules, and https://github.com/NixOS/nix/pull/5497 has languished :(

[grumbel]

You can fix that by adding '?submodules=1' to the URL. It does get rather annoying when compiling local code, as that turns a simple 'nix build' into:

    nix build  "git+file://$(pwd)?submodules=1"

But it will work with submodules.

Another mild annoyance related to this is that Flake inputs can't be expressions, this can make it a rather difficult to fetch dependencies that aren't packaged in a way understood by the existing fetch methods.

[microtonal]

as that turns a simple 'nix build' into

Which reveals another annoying thing about using flakes for development. It copies every time you build something the whole darn project into the Nix store. Fine when you have a small project, but with large projects this becomes a significant overhead.

[ParetoOptimal]

> It copies every time you build something the whole darn project into the Nix store. Fine when you have a small project, but with large projects this becomes a significant overhead.

Isn't copying the repo to a read only location just inherent to the problem of reproducibility?

[grumbel]

Not necessarily, most build systems allow you to separate source input and build output. So grabbing the source from the current directly is a possibility, if the build script is clean and doesn't do anything weird in the source directory.

In Nix you can get that via 'nix develop', which gives you a shell with access to the individual build phases and with a bit of tweaking, allows you to run them from your source directory without copying the source around first (see 'declare -f genericBuild').

[grumbel]

When you build into the store, you have to rebuild from scratch. However Nix does allow incremental builds inside your source directory as well. If you spawn a development shell like:

    nix develop .

All the build phases are available as shell functions or variables (unpackPhase, patchPhase, configurePhase, buildPhase, ...). The phases depend on each other, so if you want to skip a phase some additional tweaking or setting of environment variables will be necessary. The "genericBuild" function is the top level entry point for a normal build, the source can be viewed via:

   declare -f genericBuild

and should give a bit of an idea what Nix is doing.

[Smaug123]

Oh, you're a lifesaver. Thanks! (I spent quite a lot of time Googling how to do this, and never found it.)

[civodul]

Guix channels (not to be confused with Nix channels) let you turn a Git repo into a package collection:

https://guix.gnu.org/manual/devel/en/html_node/Channels.html

I guess it serves some of the same purposes as Flakes, and it comes with other bells and whistles such as authentication (which I think is kinda important!).

For the rest, your description of Guix is largely incorrect ("has none of that", "you have to roll back the complete Guix system", etc.), but as a Guix hacker I'd be interested in better understanding your experience and perceptions if you'd like to get in touch with us on the mailing lists!

[baby]

I’ve heard guix uses scheme, so I’m not sure I understand how it can be friendly (you (see (what (I (mean))))))

[benreesman]

You aren’t dumb. Nix is that rarest of birds: something damned-near impossible to learn well that’s still worth it.

Learning Nix well from the Internet is a horror film. But… once you do it’s shockingly, actually worth it.

Getting anything complicated working/building under Nix is a PITA, the payoff is that when you fix broken things, they stay fixed.

[setheron]

I introduced Nix into our build setup as well. We are an acquisition within Google and it's been quite a fun adventure incorporating it alongside Bazel.

Nix has been great but I've run up against a few thorny edges. Luckily I've sent out patches for them; long love OSS :)

[oxff]

> more like I am reverse engineering something rather than using it

Perfect description.

[louib]

I tried learning using several different resources, but what did it for me was the nix pills (https://nixos.org/guides/nix-pills/). I'd say it's becoming a bit outdated now with the nix standalone command, but the fundamentals are still mostly the same.

[microtonal]

Perhaps in contrast to most people's experiences, I haven't found Nix hard to learn. I think it really helps if you have learned Haskell before. Nix (the language) is really a simple functional programming language, but Nix relies quite heavily on functional programming concepts like laziness, fixed points, etc. A lot of things are underdocumented, but if you understand the Nix language well, it isn't hard to look up definitions in nixpkgs. I can understand that it is all very alien and overwhelming if you do not have a grounding in functional programming.

The primary issue for me has been that Nix is a very deep rabbit hole. You can spend enormous amounts of time on making your configuration more functional and declarative. Pretty much like you can spend enormous amounts of time on customizing an Emacs configuration. It's hard to strike a balance. And outside declaratively defined infrastructure (servers), it's probably not really worth it. I could almost fully reproduce my NixOS system (there is always some mutable state left) with a single command. But takes many months of effort to get to that point. On the other hand, I can set up a fresh macOS or Fedora Silverblue systems with all my customizations in 1 or 2 hours and have to do that maybe once of twice a year? So, ¯\_(ツ)_/¯.

I think the balance is different when you manage a lot of servers and most servers can be defined as a function with a small number of varying parameters.

The other part of the rabbit hole is that software breaks frequently in Nix. Upstreams do not really develop things under the assumption of a non-FHS, immutable system. So, you were going to work on something, but before you know it is 30 minutes later because you ended up fixing some package you need and that broke. Similarly, you'll end up packaging a lot of stuff and spending quite some time making it fit the Nix mold (looking at you Python packages that mutate in-place, sigh).

I love Nix as a principle -- it's declarative, immutable, pure, reproducible. But in practice, you can reap many of the same benefits from impure, inferior alternatives, with far less work. Yes, Docker is an ugly duck compared to Nix, but it brings 90% of the reproducibility benefits and probably everyone on your team can be up and running in hours. Rust's cargo doesn't allow you to specify every non-Rust dependency exactly, but with a Cargo.lock file you can make most of your build reproducible. NixOS is a clean, immutable, declarative system, but other systems offer a subset of its features, such as atomic upgrades/rollbacks, immutable root, and isolated applications (e.g. Fedora SilverBlue + Flatpaks, Fedora IoT, macOS). These alternatives are far more familiar and easier to work with. You can get most (but not all) of the benefits of NixOS with far less work.

Worse is better.

[crabbygrabby]

I know Haskell, regularly write in rust for fun, and nix is by far the most complicated thing I've tried to learn hahaha. Glad you were able to figure it out, and seriously thank you for sharing your opinion of how you view it even after learning it.

I see some younger folks really investing time in it and it worries me. Sometimes smart people fall under this fallacy of "you have to be smart to learn this so it must be good because it's hard" when really it looks more like quicksand than asphalt. Quicksand is hard to get out of, but there's still no reward for falling in.

[kaba0]

I recommend looking at already existing package descriptions for programs written in the same language you are looking for. Chances are, you can hit it enough times to make it package up your program as well.

I don’t think learning nix from its internals is worth it, especially not for beginners.

[pxc]

> I don’t think learning nix from its internals is worth it, especially not for beginners.

Nitpick: what you're calling 'Nix internals' are actually Nixpkgs internals.

FWIW, I think that to some extent this is just the way you have to go when you're trying to quickly take up a new (to you) configuration language for practical purposes.

I've somewhat recently started putting together some simple CI/CD pipelines at work with a tool called Dagger, which is a container image builder-and-runner based on BuildKit and the CUE language.

When I first started learning Nix years ago, I had a lot of packaging trouble with some quirky software that wasn't yet packaged for Nix, and I only ended up getting unstuck by asking for help on IRC. One generous person was hitting me with incredibly helpful links to examples from Nixpkgs and the documentation left and right, and I was astonished with their fluency. I asked them how they knew all this, and they basically said:

> Idk, one day I just sat down and started to read through some of the Nixpkgs codebase, and now I `grep -R` through it a lot to look for examples.

At the time it sounded adventurous and crazy to me, as I was pretty intimidated by the prospect. Eventually that became one of my standard tools: I treat the Nixpkgs repo as another source of documentation.

Fast forward to today and my Dagger implementation efforts, and things got immensely easier for me the moment I started treating Dagger like Nix. I cloned the upstream Dagger repository and started searching through the unit tests with `rg`, and all the answers to my questions about what functionality Dagger exposes via CUE modules were right there. Tons of the guesswork disappeared, even though I don't yet have a great grasp of the CUE language.

In some ways, the Dagger case is harder, though. In the name of simplicity and partly for other language design reasons, CUE is emphatically not Turing-complete. This is interesting and attractive in some ways, but one of the consequences is that sometimes you hit bottom in the CUE libraries to find some compiler directive that links in functionality which had to be implemented in a Real™ programming language (in this case, Go). So to really understand, e.g., the custom data types Dagger uses for secrets, you have to go read code in another language.

By contrast, in the Nix case, the language is powerful enough that you can learn everything you need to for even very advanced Nix usage only by reading Nixlang, either in Nixpkgs or in other examples. You never actually have to dive into true 'Nix internals' (the Nix repo, which is C++ code) to figure out how something in a Nix example is achieved. With a more restricted DSL, you sometimes do.

Maybe one valuable 'missing manual' would be something like 'How to read Nixpkgs'! It's such a rich source of examples. I think making earlier reference to it could help a lot of people make faster progress and feel more effective when learning Nix.

[yusefnapora]

I would love to see a "guided tour of nixpkgs," using real-world packages as examples and building up in complexity as you go.

I feel like so many of the problem I encounter as a new nix user have been neatly solved already and there are probably tons of great examples in nixpkgs, but I have no clue where to look, or even what to grep for.

It would be super helpful if there were a resource with examples of common problems and how they're solved in the real world. For example, say you're packaging something whose unit tests hit the network. There could be a link to a package definition in nixpkgs that overrides the check phase to disable the problematic test.

[AndersonTorres]

When anything hits the network, there is nothing to do besides

- disabling the code that hits the network, whether by patching or by configuration ("--disable-downloads" or anything similar)

- or emulating it; e.g. if the script downloads a file, we can `fetchurl` it and move the downloaded file to its expected place. 9Of course, it works when the script does not override the downloaded file; otherwise, go to the previous option)

Two examples came to my mind:

- Arcan vendoring required patching its cmake scripts, because cmake does not honor the cache and tries to download things anyway:

https://github.com/NixOS/nixpkgs/blob/901978e1fd43753d56299a...

- cardboard didn't require it because meson honors the cache (and Nixpkgs configures Meson to not download anything):

https://github.com/NixOS/nixpkgs/blob/901978e1fd43753d56299a...

[benreesman]

I think the first step to solving problems is acknowledging them: lazy? cool. duck-typed? cool. weird thunk stack traces, eh, workable.

Lazy and pure and duck-typed and broken ass stack traces? Fuck that. Fuck that in particular.

Eelco fucked up. Badly. nixpkgs uses Haskell type signatures as comments in the core ‘lib’s, because duck-typed pure lazy functional is fucking insane.

[rkrzr]

I agree - Nix, the language, really is at the root of many of Nix' problems.

The bad error messages, the useless stack traces, and the overall un-debuggability of Nix are not just minor issues. These are at the core of the steep learning curve of Nix. These are also the reason why with Nix you are often 'dead in the water' and don't know how to fix a problem, and end up googling for hours or asking somebody else who knows more about Nix.

And these issues cannot be fixed by having "better docs" or a better tutorial.

If somebody were to rewrite the language today then my suggestion would be to make it statically typed, like Haskell. I would probably also make it strict, with optional laziness, although that's less important imo. And providing good error messages and good debuggability should be explicit design goals.

Making it statically typed would necessitate a different overall design, since many of the dynamic tricks would not be possible any more, but I suspect that you would end up with a better overall design in the end.

[benreesman]

I think you would just make it Haskell. I don’t know the exact chronology of Nix (the language), so I can’t comment with full confidence, but Real World Haskell (written by my utterly brilliant former colleague Bryan O’Sullivan /humblebrag) was published in 2008, and the Haskell described there has monad transformers and all kinds of nifty stuff. I think Eelco’s thesis was published around the same time? It’s possible that my dates are off and they really needed to rig up their own System F machine, but I suspect it just seemed cool and now we’re stuck with it.

Nix (the language) stomps like JSON or YAML for writing config files, it’s actually not bad at that, but it’s a disaster in its contemporary role.

[AndersonTorres]

2006, to be accurate.

https://researchr.org/publication/Dolstra2006

More recently there is an attempt to run Nickel as a successor for Nix: https://www.tweag.io/blog/2020-10-22-nickel-open-sourcing/

[Smaug123]

Oh thank God, I thought I was the only one. A while back, I posted "a lazy dynamically-typed language (in my book) perfectly combines the run-time reasonability of Haskell with all the development-time safety of LISP" on the Discourse, and the reply "I love the lazy functional minimal expression-oriented JSON-like packaging DSL that is nix" got 13 Likes. I thought I was going completely insane.

[benreesman]

You’re not crazy. Duck-typing a lazy functional language in a zillion KLOC repo with broken stack traces is crazy.

At my work those “O RLY” meme book covers are in vogue this week, and my last one was “Nix Haskell Overlays for Fun and Murder: Homicidal Ideation for the Working Hacker”.

It was my job to get Clash (the VHDL/Verilog synthesis tool on Haskell) building under Bazel via Tweag’s rules_haskell and playing nice with all our other Haskell shit. That’s 14 hours straight I’m never getting back.

But… because it’s Nix I never have to do that again. Even if I upgrade everything else I can keep nixpkgs at that version in my flake.lock and that will always work forever. So I’m cool with it.

[Kaze404]

> But… because it’s Nix I never have to do that again.

This is the killer feature for me. My use-case for Nix is vastly different from yours, as I use it for centralized declarative configuration through NixOS, but I have an immense sense of satisfaction knowing that hell can freeze over but I'm still only a single command away from my exact setup running on (almost) any machine on the planet.

This might not matter much for most people, but making my machine work for me is something that took me years, and I can sleep well knowing that it's finally safe under Nix vs. multiple different configuration files for multiple different programs scattered across my filesystem.

[benreesman]

Our use cases might not be that different. Whether it’s Mac laptops, physical dev machines, or cloud instances, our domain/scale is such that our machines are “pets” rather than “cattle”, i.e. they have names.

Making NVIDIA drivers work with either Xorg on someone’s desk or CuDNN on the GPU boxes with like two friggin lines in the NixOS stuff is like, doing drugs or something in terms of sheer feeling great after years of chasing that shit all around Ubuntu’s broken-ass standard of living.

[pxc]

> Nix Haskell Overlays for Fun and Murder: Homicidal Ideation for the Working Hacker

good god pls share the link

[benreesman]

Haha sure. I don’t know anything about image hosting on the fly so pardon the sketchy ass link: https://ibb.co/Dr5Jsrx

[pxc]

> These alternatives are far more familiar and easier to work with. You can get most (but not all) of the benefits of NixOS with far less work.

> Worse is better.

There are some potential alternatives out there that seem inspired by Nix but which are less strict. I think a system that offers and manages Nix's purity but doesn't require it could end up the final winner some day. Shea Levy had some ideas for eventually achieving this in Nix that he called 'Nomia' (probably riffing on the name of Sander van der Burg's Dysnomia, I'd guess), but his life circumstances changed before the project really materialized. But Nix still could be the place where that is first achieved.

Meanwhile new container image-building DSLs like Dagger¹ and Bass² look a lot like a Nix which is impure by default, and bash builders are replaced with BuildKit directives. And there's Denxi³, which looks more like a Nix that lets you reduce its purity when you want to.

For now, though, the Nix ecosystem is still vastly more capable than these alternatives because of all the incredible work that Nixpkgs represents. Nix we know it today could still end up the concrete basis for what succeeds it, and not just an inspiration.

1: https://dagger.io/

2: https://bass-lang.org/

3: https://docs.racket-lang.org/denxi-guide/

[AndersonTorres]

My experience was seriously different. I started with Slackware, switched to Arch and after a failed attempt to run some AUR scripts, I was tired and switched to NixOS.

Incidentally, I treat both my Emacs config and my NixOS and home-manager configurations as Igors. They are never really finished. In this sense, the "many months of effort to get to that point" does not matter. Also, with NixOS I feel I can be bolder than with other distros, knowing with way more accuracy what is breaking my configuration.

> Yes, Docker is an ugly duck compared to Nix, but it brings 90% of the reproducibility benefits and probably everyone on your team can be up and running in hours.

The Docker replit was using before Nix was going hopelessly "uglier": https://blog.replit.com/nix

Also, what Cargo and NPM and `<insert lang here>` lockfile system do for each lang, Nix does in a more general package management level.

[rewgs]

I’ve religiously updated a new-linux-install.sh. The majority of packages I build from source, and the remaining I use the distro’s package manager. I’m well aware that nix is a very different beast, but in practice this gives me basically the same outcome. Nix is incredibly interesting to me, but all the painful stories have kept me on my very simple and long bash script.

[BiteCode_dev]

Everybody that sold nix to me said the goal was to replace the complicated side effect mess that is our current stack with something simple and pure.

As with FP, eventually, practicality beats purity I guess.

[ParetoOptimal]

> As with FP, eventually, practicality beats purity I guess.

Until you can take enough advantage of purity that it's practicality is higher.

[crabbygrabby]

Probably not worth it. Not trying to be dismissive but genuinely, it's fine to have side effects and if it saves 1000 hrs of engineering over the life cycle of a project I'd wager it's completely worth it.

[rowanG077]

The keypoint here is that nix requires a larger up front investment. But once done it will save you 1000 hrs of engineering in the long run. It's absolutely faster to start a project without nix and just start coding. But once you have many deps, multiple developers, ci, multiple environments nix becomes a no brainer.

[SuperSandro2000]

But the big problem in the end will likely be that you have no idea how reproduce the setup from scratch again

[crabbygrabby]

It's not just you, the user experience of nix is terrible. It's biggest feature of "pure" builds is mostly unnecessary given the complexity. It's runner up feature of configuring builds with a few scripts can also easily be accomplished by simpler tooling...

I don't get why it's necessary or unfortunately why it is so complicated... Anything that can be done in nix can be done in a tenth the time by any employee in a more commonly used technology. In industry, this is a very bad thing in my opinion

[benreesman]

The user experience of Nix is terrible. I’m not sure that it’s even fixable.

But Nix or something like it is going to take over the world.

Dynamic linking by default is absurdly stupid and mostly motivated by GNU politics. It’s a terrible problem. Many (most?) of the users of Docker don’t even realize that this is the problem Docker is solving for them. But they know they have a terrible problem and Docker helps a lot.

Nix is Docker on steroids. It’s Docker that got bitten by a radioactive spider.

Linux namespaces and cgroups and BSD jails have been around. Docker made it Just Work.

When someone does that for Nix, which is basically Docker done by computer scientists, it’s game over for anything else.

[lillecarl]

You can use Nix to build OCI(images) and run them on Kubernetes if you want to.

Nix is a souped up package manager, Docker is a container runtime.

Nix depends on packages existing in /nix Docker chroots into a "folder" and runs a command(+many more things).

Lets not mix technologies up for the readers too much.

[benreesman]

I’m well aware that Nix can produce container images. Xe has a great post about it.

People use Docker for a lot of reasons, but mostly? Same Dockerfile, same outcome, mostly every time. No one is moving /usr/lib/x86_64 around under you. It’s a real sea change, on the order of revision control: we hadn’t even realized that we were living with constant low-level anxiety that someone was going to break our computing environment at any moment. “sudo apt upgrade —whatever”, eh, maybe next week, we’ve got a release coming up.

Calling Nix a souped up package manager is like technically correct maybe?

It’s ‘git reset —hard HEAD^’ for your whole computer or fleet of computers. It’s utterly fearless experimentation, it’s low/zero runtime cost isolation and reproducibility.

It’s early days ‘git’ for systems: pain in the ass to learn and use, frequently and credibly accused of being too hard for mortals, but profoundly game changing.

Whether Nix per se remains the plumbing, someone is going to do good porcelain and end DevOps as a specialization, along with Docker and Canonical and mandatory glibc nonsense and a thousand other things that have overstayed their welcome. Disks are big now, we can have a big directory full of hashes. We can afford the good life.

It’s going to be a big deal.

[CMCDragonkai]

Nix is a programming language/system for software packaging. Along the way, it turns out OS configuration is also just packaging... As long as you don't have anything too dynamic.

It is the most sophisticated programming language/system for software packaging on the market today.

We use it for all our reproducible software development environments. All of our project templates. And all of our employees are issued NixOS configurations so everybody on the team has the same NixOS, the same Nix package overlay... Etc. This level of consistency ensures that we get a level of reproducibility from hardware to OS to user profile to development environment to CI/CD that just doesn't exist anywhere else atm.

It's not entirely perfect but that's not really Nix's fault. It's actually the fault of the rest of the software industry that persists with bad tech. All of Nix's complexity stems from having to package everybody else's mess. Anyway one day I hope the CI/CD systems in the world will just provide Nix jobs directly instead of proxying through a docker container image that I have to setup atm.

[benreesman]

Your shop sounds pretty ahead of the curve. “It works on my box!” “What do you know, mine too!” “Production person over here: I’m getting the same thing!”

It seems to me that people who go through the agony and ecstasy of getting it set up do so because the difficulty of the domain is high and the headcount is low: you need leverage in that setting.

Can you share anything about what motivated your group to get it dialed in?

[lillecarl]

In the more philosophical sense I think you're right. I've been using NixOS for 2 years now on all my machines* but I still don't know the language.

I use it as a stable base system, but when I need to do something that I'm unable to do with Nix I still drop into a $DISTRO container and do my work there where everything is stateful and "disgusting".

Having worked with DevOps for awhile I can happily tell you that we don't run around building Docker containers all days, at least in my team the developers do so. We provide them with a base to build upon, provide CI and stateful services like databases and storage.

I spend an awful lot of time writing Terraform and Helm charts though, since things needs to run somewhere.

But yes, the immutable nature of Nix is great. Nix was VERY helpful for me when I switched GPU's from NVIDIA to AMD on my desktop, no screen? Reboot, reconfigure, retry.

But yes, I agree. Something that resembles Nix will take over the computing world, as someone said somewhere in the comments of this post. The great thing about Nix is that when you fix something it stays fixed. Even if it was a PITA to get there.

[benreesman]

I believe that we’re in violent agreement :)

I used the exact same words “things stay fixed” elsewhere in the thread. Which is big psychologically. I’ll do some really difficult or even painful things in good humor if I get a lasting benefit. But if someone or something is just going to yank the rug underneath me next month? Fuck it, hack up some doofy shell script and call it a day.

Nix aligns the incentives on getting things right. I’m happy to learn the arcana of some weird corner case of some tool or service if I can apply that in a way that is permanent.

Now we just need someone to unfuck the UX nightmare :P

[0x457]

> Same Dockerfile, same outcome, mostly every time.

Uhm, no? Dockerfile has tons of side effects:

- doing `apt-get update -y`? On some machines it will run, on others it won't be due to caching. - Using `FROM` that isn't locked to sha256? Well, sometimes you will get version 1.2.3 sometimes you will get 1.2.5. Sometimes a new one will get tagged with the same tag. - It literally has network access during the build, unless you include a hash of what you're downloading, there is zero guarantee it will be the same download.

I think the majority of leaf containers rarely get the same result with the same Dockerfile. The only thing that is guaranteed with docker is that the same image will be the same image, but ensuring that different machines pull the same version of an image is another story.

[benreesman]

I’m being a bit generous to Docker in the above comment: I believe that this is what users are hoping to achieve and getting closer to achieving than they would otherwise. Docker is basically a roundabout way to get static linking behind Drepper’s back. Almost no one is using it to bin pack such and such CPU/RAM to the request serving process and the batch processing process given finite SKUs.

Modulus the absurdly high barriers to entry, Nix is trivially better for this purpose.

[kaba0]

I don’t think mixing up the two was parent’s point at all. He/she just made an analogy of where Nix currently is vs where should it go to “take over the world”.

[baby]

I think we’re used to tools using convention over configuration more and more, which is great, but as nix is trying to solve a different problem it is all about configuration, which makes it really hard to understand. Maybe it would help to have a GUI or a local webapp to sort of show the state of what you have and debug things… not sure.

[natrys]

I don't see how that addresses OP's problem, which is poor and confusing documentation (I felt the same too). Everytime I hit a snag and searched for an issue, there were never a comprehensive and centralized source of solution, rather a bunch of pages each with different holes in information sprinkled with outdated advices. I share the feeling that I was reverse engineering things that should just be plainly documented.

[baby]

Agree that documentation could be better, but as someone who has spent reading a lot of it I was still left with a sense of “how are things really glued together here in this nix project?”

[infogulch]

> convention over configuration

Hmm I guess the nix approach would be "convention as shared configuration"

[noidiocyallowed]

I'll translate. It's utter garbage bullshit.