Hacker News new | ask | show | jobs
by yeck 924 days ago
Nix package manager's `nix-shell` is something I wish more people knew about. Nix is gaining some popularity, but people often think of using it has to be a really big commitment, like changing your Linux distro to NixOS or replacing your dotfiles with a Nix-based one (using the Nix package manager).

What I wish more people knew was that you don't need to do those things to get value from Nix. Create project specific dev shells that install the packages (at the correct versions) to work with that project can almost replace 90% of the docs for getting setup to work on a project.

4 comments

Have you tried compiling software with a nix shell? It gets linked to the Nix store. Needless to say it was a frustrating revelation.
The missing link for me is nix-copy-closure, which I learned about from a post by mitchellh: https://mitchellh.com/writing/nix-with-dockerfiles

conceptually a game changer for me. In practice it's far from a silver bullet (because every language prefers its own package management so you still have to manage those), but when it works it's quite magical.

You can patchelf to link to the host system libraries instead, or some projects can statically compile (inc musl) with less drama than usual, since your cross compilation toolchain can be part of your nix-shell.
Isn't this what you want? You are reliably linking to the specific dynamic libraries you are compiling against.

Or was the issue that you expected them to be portable? Or use commonly known dynamic library locations?

It was just surprising, is all. When I use use <x> application from a nix shell, it pretty much always works the way I think. The compiler experience was very jarring, but yes I understand why it works the way it does.

I was more or less pointing out the UX issues with Nix that end up turning many people away.

There is definitely a learning investment in order to write good Nix expressions. But, if you write a good nix shell expression for your project, other devs should be able to jump in without really needing to understand those Nix expressions and still get a working environment.
Sometimes it matters, many times not.

For example I tried to run pip install yesterday on MemGPT on Nix.

It failed with a C++ error because they use miniconda.

I just created a nix shell with python, pip, etc and ran the pip install command.

Things work fine.

Oh God miniconda is a horrible piece of software on Nix.

I fell down the Nix rabbit hole, and miniconda was one of the worst things to get working. My first pass used an FHS environment, but eventually I just got the environment.yml file working in micromamba and used that instead. Except micromamba ships it's own linker that I had to override with SHAREDLD, or some random python c++ dependencies wouldn't compile correctly.

I love Nix, but my list of complaints is a mile long. If you want to do anything with opengl in nix, but not on nixos, just give up. NixGl just doesn't really work.

Good luck getting something like Poky (Reference project for Yocto) running in Nix. The only working example puts it in an FHS environment, which uses bubble wrap under the hood. But then because you're in a container with permissions dropped, you can't use a vm. The solution I see in the support forums is roll your own alternative FHS environment based on something else.

/Rant

Yes, this is where I am at. Used it for over a year in a DevOps role and have developed a huge distaste for it. Despite the language itself being one of the most complained about things, I didn't mind it so much. It was the mile-long stack traces, which were often wrong, and constantly fiddling with things I didn't want to fiddle with to get something working. Just ended up costing me way too much time.
Although I haven't really used it, jetpack.io's Devbox [0] is an abstraction over Nix for the usage you describe.

[0] https://www.jetpack.io/devbox

I've played around with it a little bit, but not enough to make any judgements on it. Something like devbox could be the sort of thing to make nix-shell accessible enough to see wider adoption.
Snap also has a way to open a shell inside the context of a snap package
It's good for a c or c++ project where libraries are very environment specific. But most modern languages have their own package/environment managers which makes Nix redundant.
Not really. I introduced it to our Python projects at work and it's been great. Partially because of poetry2nix, and partially because it makes it easy to include other stuff like a specific version of Redis for testing purposes. Everybody gets the exact same dev environment, reducing a ton of "works on my machine".
Presumably it also can fill the role of conda/mamba i.e. also managing C/C++ libraries in the same way in the nix environment, isolated from the system libraries?
Yep, it can lock down exact versions of those libraries as well, which is great for not mucking about with lib versions between even different Ubuntu versions, not to mention distros or macOS.
Except nix doesn't support Windows, which is best reason to use conda (having to support Windows).
virtualenv is the python way. For things like redis and other external web stuff, docker is the standard.
Sure, that works. Or I can have it all in a single shell.nix file that covers everything and is super simple to use. It's great for handing off to coworkers that don't usually use Python.
It's not simple. The nix programming language is like untyped ML. Most people aren't used to it and even if you are familiar with it it gets hella hard to read. Learning curve is huge.

One docker file and a poetry file works just as well. And is simpler. It's literally the same thing but using os primitives to manage the environment rather then shell tricks. Makes more sense to me to use a dedicated os primitive for the task it was designed to be used for.

Additionally docker-compose allows you to manage a constellation of environments simultaneously. This is nowhere near as straightforward with nix.

I love nix but being honest here. It's not definitively the best.

The biggest reason right now to avoid it is adoption. Most people won't know what to do with a shell.nix

> One docker file and a poetry file works just as well. And is simpler. It's literally the same thing but using os primitives to manage the environment rather then shell tricks. Makes more sense to me to use a dedicated os primitive for the task it was designed to be used for.

1) not just as well because docker is repeatable, not reproducible

2) not if you need GPU acceleration which is a headache in docker, but not Nix shells

> Additionally docker-compose allows you to manage a constellation of environments simultaneously. This is nowhere near as straightforward with nix.

- devenv.sh - arion - https://flakular.in/intro

> Most people won't know what to do with a shell.nix

The same was once true for Dockerfile

>It's not simple.

Neither is using virtualenvs for Python packages with native extensions.

nix is far simpler for consumption. My coworkers don't like fancy new things, and they haven't had any complaints. They don't have to dick around with half a dozen different commands to get everything set up, or bother with docker volumes/port mapping/etc. They just run nix-shell and it all works. That's all you have to do with a shell.nix file, it's very simple.

It is harder to write on average atm, but it's very much worth it to me when it comes to sharing code for development. Also, LLMs help quite a bit when writing nix.

Most language package/environment managers do not come close to giving you the guarantees that Nix does.
Two problems:

a) Unless you literally write everything in one language, you will have to deal with learning, supporting and fixing bugs in N different package/environment managers instead of just one.

b) If you have a project that uses several languages (say, a Python webapp with C++ extensions and frontend templates in Typescript), then Nix is the only solution that will integrate this mess under one umbrella.

a. Using nix in place of a package manager means dealing with libraries specific to that language. It's still managing different apis. And more potential for bugs and unforeseen problems in custom 3rd party APIs as opposed to the official one. Admit it, you hit tons of problems getting everything to work fine with python.

b. C++ is the only one that would benefit from nix here because C++ dependencies are usually installed externally. There's no folder with all the external sources in the project. Even so this can be achieved with docker. If you want you can have docker call some other scripting language to install everything if you want "one language" which is essentially what you're doing with nix.

a. you hit tons of problems getting everything to work fine with python - of course, but the maintenance burden is an order of magnitude lower than integrating all this manually.

b. No, docker is not a solution. Docker is another problem and a separate maintenance nightmare.

(Nix solves maintenance problems at scale, Docker explodes them exponentially. I would not ever recommend using Docker for anything except personal computing devices you don't care about.)

I agree.

There are many python packages that have other dependencies not managed by Python package management. The pain of figuring out what those implicit dependencies are is effectively removed for users when configured as a nix shell.

I had to use it for a c++ project and it was one of the biggest waste of time and frustrating moments of my computing career, there were constant breakages due to glibc mismatches, Nvidia drivers and whatnot, and getting an host IDE to have semantic understanding of the paths , etc... necessary for normal completions and stuff was nigh impossible.
Yeah but other than conan it's one of the few things where you can get a sort of "project package manager" experience like npm with C++. It's not nearly as user friendly as what they have for python or nodejs.
No way. Language specific managers are terrible at managing external dependencies. Trying to get python packages to link to system libraries is terrible. Nix makes it infinitely better.