[WiseWeasel]

By “vendoring” dependencies, I’m guessing you are talking about pulling them out of node_modules (into a js/vendor dir), and checking them into your repo, repeating the process every time you want to update. I will make an offering to Cthulhu if it’ll ensure we won't be using webpack and node 10 years from now in order to build those old snapshots without considerable effort.

[danShumway]

> pulling them out of node_modules

No, even further. Just commit your node_modules folder.

There's this idea that vendoring has to be really complicated, but committing node_modules was actually official advice early on in Node's history. And what that means is that if you clone your repo, you just run `build`, you don't have to change or configure anything at all. The only dependency you have in your project becomes a node binary -- and honestly, you can vendor that someplace as well, node is reasonably portable.

There are a lot of advantages to this, not the least being that if you're demoing or working on a project someplace without an internet connection, having an up-to-date Git repo is all you need, you just have your entire project. If you switch branches, your project is ready to go.

Of course, if you have something like 1G of dependencies, checking them into Git starts becoming kind of cumbersome. But if possible, avoid that, for the same reasons you would avoid JS dependencies that compile native binaries. Having gigabytes of dependencies is going to be a nightmare for debugging and security, no matter what language you're in.

But I have built some pretty complicated projects where vendoring is fine. It doesn't balloon the size of my git repo, it doesn't make the project less portable, and it regularly saves my butt when I forget to check out the right branch before getting on a train without an Internet connection.

And all you need to do is not add node_modules to your gitignore.

[filipo]

Not a node dev myself but why not simply reference dependencies with fixed version? According to semver (that is probably what most node projects use anyway), you are free to use latest minor and patch revisions and leave the major fixed. Shouldn‘t this prevent most compatibility problems?

[danShumway]

> why not simply reference dependencies with fixed version?

This is the second best strategy, but:

- node makes it kind of annoying to do this, because unless you do a clean build every time (which is much slower) `package-lock.json` will get largely ignored.

- I have seen multiple projects break with semver. Just because people are supposed to avoid breaking changes in minor releases doesn't mean they actually do.

- Even if you're downloading the same version, there are scenarios where natively compiled binaries can bite you. I've run into issues where node-gyp just wouldn't work unless I downloaded a completely separate compile toolchain onto Windows and restarted the computer. If you can avoid that and have all of your dependencies be pure Javascript, you will eventually save yourself a lot of horrible dev-ops work when the "quick" install on a new VM turns into an hour long process of watching Visual Studio tools download (tools that to GP's point may not be available or compatible 10 years from now).

- And finally, sometimes you want to do work across multiple branches without an internet connection, and vendoring your dependencies allows you to check out a branch and know you have the correct dependencies even if you don't have a connection to run `npm install` with.

[filipo]

Ok I see; thank your for the explanation. So out of interest, I did some research and found that approx. 30% of packages depend on native implementations further down... so this seems to be a huge problem...

Additionally I looked at the sqlite3 package and not too surprisingly it uses node-gyp as well. So even checking in node_modules might cause the problems you described.

Out of interest: which backend language do you recommend to reduce such problems...?

[danShumway]

I use Node. I don't think Node is wildly better than anything else, but I'm used to it and it's good enough, at least for now. I'm unaware of any back-end language that forces dependencies to stay in the same language, but something probably exists somewhere.

I agree that node-gyp is a problem. It's possible to avoid, but can take some work (particularly when you start dealing with databases that basically have to be in compiled languages). The one saving grace is that if you have a set hardware that you're running on, you can sometimes vendor node-gyp dependencies. Unfortunately, that comes with some weird edge cases, and sometimes your dependencies break because your OS changes. I don't recommend it, but I understand that sometimes it's unavoidable.

I am hopeful that native WASM will solve some of these problems, but I don't know how realistic that hope is.