[jrochkind1]

> Inline all dependencies during publish… From a maintainer’s point of view, the upsides are clear: you get faster boot time, smaller downloads, and — as a nice bonus — no bogus vulnerability reports from your users.

And when you inlined a version that later really does have a vulnerability, it is not easily flagged or fixed by your consumers.

The tension between "upgrades (especially of indirect dependencies) might break" and "upgrades (especially of indirect dependencies) might be necessary to fix bugs or patch security vulnerabilities" is real. There is no magic bullet to get around it. There are practices to try to balance it -- which generally involve ecosystem-wide commitment to backwards compatibility, reflected in semantic versioning (and minimizing major releases).

That npm dependency trees are often insane doesn't help though. I'm not totally sure why they are so insane, but I increasingly think that npm's ability to have multiple versions of same dependency in the dependency tree -- often seen as a huge advantage over other platforms -- is in fact part of the problem. It makes the dependency tree even more insane, and it also accomodates a false belief that it's fine to lock to very specific versions of dependencies -- because it won't prevent other dependencies from co-existing in tree with other conflicting requirements after all. Which then accomodates a false belief that dependency maintainers don't need to worry too much about backwards compatibility, after all consumers can just lock to specific working versions... and now we wind up with insane dependency trees which depend on vulnerable versions in ways that require a whole bunch of releases to resolve.

[theptip]

> I'm not totally sure why they are so insane

I think a big part of it is that due to much stronger pressure on bundle size than most other environments, each library tends to be small, so there have to be more to carry the same amount of functionality.

Duplicates are certainly a contributing factor as well, and small bundles compound with allowed-duplication to further increase the tree size. I think that small package size also probably makes it harder to require a single version for each dep, since there are going to be more edges in the graph and therefore more relations for library maintainers to keep track of (including what would in other languages be intra-package requirements), so you're more likely to get version incompatibilities.

[munificent]

I agree with all of this. Also JavaScript's "standard library" is nearly nonexistent (or at least was when Node first got big). That built a culture of people assuming they needed to reach for third-party dependencies for nearly everything (see: leftpad).

[enumjorge]

Slightly related to the lack of a standard library is that a lot of these 3rd party packages come from random people in the community. It’s great that people are so willing and able to share code, but it also means that as a community we put a lot of trust into code that may not be vetted or funded properly. I think we assume that because these packages are open source that someone is making sure they are safe to consume, but because there’s so many of them it’s hard to verify them.

[ipaddr]

You only support corporately funded open source?

[enumjorge]

That’s not at all what I said. C/C++, Python, and Rust are examples of languages that are not owned by a single company yet they are funded enough to be able to provide a stable standard library.

[tshaddox]

> That npm dependency trees are often insane doesn't help though. I'm not totally sure why they are so insane, but I increasingly think that npm's ability to have multiple versions of same dependency in the dependency tree -- often seen as a huge advantage over other platforms -- is in fact part of the problem.

Aren't npm dependency trees so large because JavaScript doesn't have much of a standard library? And also, similarly, the community is so large and has been moving so fast that even de facto standards have difficulty forming and surviving at a large scale across the community.

[handrous]

The lack of static typing (in base JS, at least) also makes it hard for tools to automatically spot very basic brokenness in dependencies without (repeatedly) running & testing the code. This makes even "safe" version bumps less trustworthy and harder to audit, and makes it harder for developers to notice if they've accidentally changed an interface on one of their libraries that they marked as a minor patch (i.e. the errors are both harder to check for, and more likely to occur, basically because they're harder to check for), so it's tempting to stick to old versions longer.

Add to that everything else—the fast pace of changes, javascript "culture", the weak standard library, the tendency to patch in what ought to either be basic language features or else avoided in favor of more-vanilla idioms, often in competing and incompatible ways—and all that is how you end up with 20 slightly-different copies of the same damn library in your dependency tree, and then 20 other copies of another library that does the same thing.

[zelphirkalt]

Big part of the insanity is also how projects introduce dependencies for very simple things like padding a number or a string.

[HideousKojima]

https://www.npmjs.com/package/is-odd

447,211 weekly downloads for what can be done in vanilla JS with foo % 2 === 1;

[oauea]

You can thank Jon Schlinkert for that: https://github.com/jonschlinkert?tab=repositories

He abandoned the projects, but:

* https://github.com/jonschlinkert/is-odd

* https://github.com/jonschlinkert/is-even

Of course that doesn't stop them from getting ~500k downloads/week.

[Ayesh]

Which itself depends on is-number package.

[zelphirkalt]

You and GP have shown me new even more scary depths here. Not sure whether I should thank you for that. (When does the next flight off this rock go?)

[tshaddox]

It's not really that crazy. This package is a small amount of code, but it's important code (the same goes for this package's dependency is-number). This package shows up in the dependency trees of some popular packages, which is probably where most of the weekly downloads come from.

If you're writing straightforward application code where you already know you have a valid number, then this package probably isn't for you. You can just do num % 2 === 1.

[bscphil]

Well said.

I would add that I think the devDependencies solution is underrated. Not using it is a bad practice. npm has a nice feature, `npm prune --production` which will remove all the dev dependencies for you resulting in a clean build of the program. You can easily have things set up so that none of the development dependencies that have all these audit issues are ever present on your production machines if you do things right.

Furthermore, if you have a project with end users, devDependencies allows you to make clear in your issue template that audit issues that don't show up in a production build are very likely to be false positives and will probably be closed without comment. If you aren't properly isolating your dependencies, you can't take advantage of this.

At the end of the day, as you say, the issue is largely with Node projects having too many dependencies and a large number of relatively minor issues that affect very specific use cases get reported. That's a hard ecosystem problem to solve, not necessarily something that indicates an inherent problem with npm audit; but if someone isn't using devDependencies, that alone could constitute an enormous improvement in their workflow.

[Vinnl]

If there's a vulnerability in Webpack (a devDependency) that injects malicious code into your bundle, `npm prune --production` won't save you.

[remram]

This is not a vulnerability (ie. security bug) it's an attack (ie. malicious).

[Vinnl]

It doesn't really matter how you call it; the problem is that there could be CVE's in your devDependencies that affect your production build, and pruning those dependencies after using them to create that build doesn't remove the risk.

[hinkley]

I'm currently using 'npm ci --prod', which means that they never get installed in the first place.

However it's incompatible with optionalPackages, so I'm carrying around some tertiary dependencies that are a small but noticeable fraction of the entire archive size.

[twistedpair]

> That npm dependency trees are often insane

For my hundreds of repos (Java, Scala, JS, Typescript, Python...), Snyk flags 99% of the CVEs for the JS repos. Shocking how I've only seen a few dozen or so Java based CVEs flagged over the last few years.

Perhaps it's because my NPM based repos have ~10K more dependencies? That and the Java stdlib handling most needs w/ the vanilla lang.

[hinkley]

There is no magic bullet, no, but a lot of people are confused about how much less sense npm lockfiles make today than ruby lockfiles made when npm was still new.

Some of us are considerably salty about it. Especially the design-by-PR aspects of the whole thing that have resulted in confusing gyrations from one version to another.