[pyre]

> I've gotten to where I only install stuff via npm when I'm on a free connection; I normally work on mobile broadband, which is very expensive

You could always use this:

https://www.npmjs.com/package/npm-proxy-cache

It caches the package listings and the packages that you download. It will act as a pass through that with a limited TTL on the cache, but there is an option to fallback to the cache if you can't connect to upstream.

Granted, you have to have already installed something for it to work as an offline cache.

Also, part of the problem with all those files is that npm allows packages to installed pinned dependency versions. If package-a requires lodash 2.x and package-b requires lodash 3.x, then both will be installed within the respective package's directory. For example let's dive into the node_modules/ in one of my projects.

  $ ls node_modules/**/lodash.js
  node_modules/cordova-lib/node_modules/lodash/chain/lodash.js
  node_modules/findup-sync/node_modules/lodash/dist/lodash.js
  node_modules/findup-sync/node_modules/lodash/lodash.js
  node_modules/globule/node_modules/lodash/dist/lodash.js
  node_modules/grunt-contrib-less/node_modules/lodash/dist/lodash.js
  node_modules/grunt-contrib-less/node_modules/lodash/lodash.js
  node_modules/grunt-contrib-watch/node_modules/lodash/dist/lodash.js
  node_modules/grunt-contrib-watch/node_modules/lodash/lodash.js
  node_modules/grunt-curl/node_modules/lodash/dist/lodash.js
  node_modules/grunt-curl/node_modules/lodash/lodash.js
  node_modules/grunt-legacy-log-utils/node_modules/lodash/dist/lodash.js
  node_modules/grunt-legacy-log-utils/node_modules/lodash/lodash.js
  node_modules/grunt-legacy-log/node_modules/lodash/dist/lodash.js
  node_modules/grunt-legacy-log/node_modules/lodash/lodash.js
  node_modules/grunt-legacy-util/node_modules/lodash/lodash.js
  node_modules/grunt-ng-constant/node_modules/lodash/dist/lodash.js
  node_modules/grunt-ng-constant/node_modules/lodash/lodash.js
  node_modules/grunt-protractor-runner/node_modules/lodash/lodash.js
  node_modules/grunt/node_modules/lodash/lodash.js
  node_modules/jshint/node_modules/lodash/chain/lodash.js
  node_modules/lodash/chain/lodash.js
  node_modules/phantomjs-prebuilt/node_modules/lodash/lodash.js
  node_modules/preprocess/node_modules/lodash/lodash.js
  node_modules/protractor/node_modules/lodash/lodash.js

That's 24 copies of lodash.js installed that could all be a unique version of lodash used only by said module.

[SwellJoe]

This seems like maybe a really big area for evenagelism in the node community. I've watched a ton of talks and tutorials lately, and several of them made a strong point of saying, "use versioned dependencies" because libraries aren't practicing good semantic versioning, so even minor version changes can be breaking. So...maybe there needs to be a lot more attention being paid to semantic versioning being used religiously.

I'm new to this ecosystem, so I'm definitely not an expert, but it's certainly been an intimidating point for me; maybe the most difficult thing to wrap my head around. I'm used to being able to spelunk into my project, and read everything I'm depending on, or at least skim it and kinda grok where things happen. How would one even do that with 53,000 files? How can anyone trust any application they build with these tools? I mean, the security implications alone are breathtaking, to me.

[vmasto]

You're being unfair. You're obviously using npm 2.x, we've since moved to 3 for a very long time now where the dependency tree is flattened and this issue is avoided.

[ksherlock]

It's better but it's still an issue when multiple version of a module are needed.

[pyre]

That node_modules directory was created using Node.js v5.10.0 and npm v3.10.5.

[Scarbutt]

Does the 24 copies all get bundled into the final JS file in the case of a browser app?

[vmasto]

Of course not.

Also the modules he lists here are mostly for tooling purposes.

[FLGMwt]

Are you sure? I could see multiple downstream versions of utility libraries like lodash and jQuery used in runtime dependencies.

Maybe not as bad as 23, which doesn't seems as likely for non-toolong stuff, but I'd still expect some divergence.

[prodigal_erik]

The problem isn't pinning, the problem is that it's routinely accepted for dependencies to disagree about which version of a library is ready for production use, because npm doesn't treat that as a disaster that should block deployment until the community coordinates their acceptance testing.

Basically, if lib1 and lib2 each use lib3, I don't want to upgrade anything until both lib1 and lib2 agree that a newer version of lib3 works.

[mason55]

What if there is a breaking change in lib3 and only lib2 upgrades? Does that mean lib1 can just never be used again because of something lib3 did?

[prodigal_erik]

If a new version of lib2 moves to a half-baked version of lib3, I want to stay on the old version of lib2 (and lib3) until the community agrees that the lib3 change is good and everyone has migrated to it. Having two different versions of lib3 in process is begging for bewildering bugs, and it was a serious mistake for npm to let that happen (much less by default!)

[woah]

While I'm not defending running multiple versions of a library, I've never seen it cause bewildering bugs, or any bugs at all. The deeply nested style of npm may have issues, but it makes it really easy to avoid thinking about dependency problems.