Part of that is just the monstrous number of files involved. For example, one of our projects has 48 dependencies, which installs 24,421 files under node_modules. NPM could probably benefit from managing each dependency as an archive.
We do atomic deploys and try to make them reproducible, so for each deploy we do a fresh install from npm-shrinkwrap.json, but even when all the modules are in NPM's local cache it's very slow at copying everything.
Unfortunately, NPM doesn't version the node_modules folder (a package becomes ./node_modules/mypackage/...)), so you can't reuse it. The NPM cache is versioned ($cachedir/mypackage/3.23/...), but can't be used directly. It would be much better to skip the cache altogether, and have node_modules embed version strings (./node_modules/mypackage-3.23/...). Then you could easily share the folder across builds.
NPM is also pretty brittle. We frequently have deploys fail because of transient network errors (repository timing out or similar) that cause NPM to fall over. The dreaded mysterious "npm ERR cb() never called" error still hits us weekly.
(Speaking of reproducible builds: NPM lets people unpublish packages. Sometimes old versions just disappear, presumably because they were unpublished. 6 months later you want to deploy a certain app, and you find it depends on some package X, which deep in its dependency graph relies on package Y 0.3, but 0.3 is gone from npmjs.com, so you have to upgrade for no reason at all.)
Have you used any other package manager? I'm not completely sure if it's just Node's dependency hell, but npm feels so incredibly slow compared to Cargo or bundler or something.
Never profiled it but I would guess that is largely due to the npm's main problem: It doesn't locked dependency versions. As a result it probably builds the dependency graph a lot more than it should.
For me this is the main thing I'd like to see fixed. Not only does it build more than it need to, it can result in broken builds. When my dependency doesn't update, I don't expect its dependencies to update (especially when I don't have control over setting the versions of the dependencies of dependencies). This has bitten me more than once.
I work on a fairly large frontend project where everything is modularized. So we have maybe 50+ dependencies that are our own. These constantly change. I detest changing branches and having to run npm to get the correct version of stuff. Takes minutes before it even starts installing packages.
Wow. I think to myself "npm is slow" several times every working day, no exaggeration. I am currently on HN because npm has been busy for about 5 minutes. Maybe we use npm very differently.
Part of that is just the monstrous number of files involved. For example, one of our projects has 48 dependencies, which installs 24,421 files under node_modules. NPM could probably benefit from managing each dependency as an archive.
We do atomic deploys and try to make them reproducible, so for each deploy we do a fresh install from npm-shrinkwrap.json, but even when all the modules are in NPM's local cache it's very slow at copying everything.
Unfortunately, NPM doesn't version the node_modules folder (a package becomes ./node_modules/mypackage/...)), so you can't reuse it. The NPM cache is versioned ($cachedir/mypackage/3.23/...), but can't be used directly. It would be much better to skip the cache altogether, and have node_modules embed version strings (./node_modules/mypackage-3.23/...). Then you could easily share the folder across builds.
NPM is also pretty brittle. We frequently have deploys fail because of transient network errors (repository timing out or similar) that cause NPM to fall over. The dreaded mysterious "npm ERR cb() never called" error still hits us weekly.
(Speaking of reproducible builds: NPM lets people unpublish packages. Sometimes old versions just disappear, presumably because they were unpublished. 6 months later you want to deploy a certain app, and you find it depends on some package X, which deep in its dependency graph relies on package Y 0.3, but 0.3 is gone from npmjs.com, so you have to upgrade for no reason at all.)