[nirvdrum]

If you, as an attacker, are going to make a fraudulent commit and change package.json, you could just pull in a bogus dependency or add a new script/command that gets run on the build/dev machine. I agree the bar to submitting a fraudulent commit is unfortunately too low for many teams. But, this also extends to package publishing, too.

We've seen packages published through compromised dev keys or the maintainer granted a new bad actor rights to publish. Here we could envision a situation where someone reports a legitimate, but low impact, security issue and also publishes a bogus package that everyone is now upgrading to. On the whole, these messages aren't encouraging people to upgrade to a new, verified release (i.e., a fixed version). They're encouraging people to upgrade to the latest release. Even if you suspend the bad actor thought experiment, the latest release of a package in all likelihood hasn't been audited beyond ensuring the previously reported security issue has been addressed. Upgrading indiscriminately is a risky activity as well. I don't think we should be encouraging people to do that (in any language ecosystem).

It's not lack of imagination of attack vectors at play here. Treating extremely low risk factors as if they're high priority reduces trust in the system as a whole. Getting developers to care more about security is a laudable effort, but I think the `npm audit` approach (as reported in the post) is going to encourage bad practices.

[trunnell]

I agree with all of your points except for the very last one.

Are we shooting the messenger (npm audit) here? Seems like the problem is lack of trust and lack of information about security of dependencies. Npm audit is just pointing that out. The size of the problem makes it very uncomfortable.

[nirvdrum]

I think where we disagree is I don't think npm audit is pointing out useful information. Or, at least, it's including a lot of useless information that makes finding the useful information hard to see. Now that it's the default when installing packages, most JS developers are going to have to deal with it. Given a lot of the data is not applicable, unnecessarily repetitive, and has severity mismatch with reality, I think most people are just going to ignore the output. If not that, they're going to blindly update packages and when that doesn't work due to dependency chains and such, they're going to start saddling open source maintainers with busy work. The net result is that when the next real mass security issue hits, it's going to get overlooked or ignored with all the other noise.

What could have been a useful tool for CI is not going to be enabled because who wants their build failing for a week while trying to orchestrate a mass dependency upgrade? I just took a look a Create React App (CRA)-based I have and "yarn audit" indicates 2,288 packages were scanned with 337 vulnerabilities (that's pretty good, I've seen it list over 500 vulnerabilities in the past). This is an app using CRA, with Relay as a code generating GraphQL client, and Antd as a component framework. There's also supporting tooling such as TypeScript, ESLint, and Prettier. Beyond that, very few dependencies are being used. Almost none of them are actually shipping in the product, whether that be due to them being build-related tools or because tree-shaking just strips most of the library out. It's a SPA hosted in CloudFront, so even DoS attacks have a very different profile than products where the server is rendering views. In this particular case, there's no shared user-generated content and XSS mitigations are in place, so the best a customer could do is DoS themselves.

I try to be very diligent about auditing dependencies before upgrading, but I can't reasonably extend that to transitive dependencies. 2,288 packages is an absurd number and this app isn't even doing all that much. I believe it's quite likely that it'd be impossible to get the npm/yarn audit report down to 0, given how quickly the JS ecosystem moves -- upgrade one dependency, deal with whatever API changes are needed, and now another has a vulnerability of some sort listed. It's certainly a burden on a small development team.

Just to be clear, I'm not advocating sticking your head in the sand and pretend everything's fine. I'm saying when you have vulnerabilities listed as high severity and you've verified they're not problematic in a project, then you've addressed the issue. But, now you have a tool that you need to satisfy and if you don't then you're going to miss the next security issue because CI shut off the "npm audit" check.