Hacker News new | ask | show | jobs
by nonameiguess 1814 days ago
It isn't broken by design. I sympathize and understand it's annoying to have to comb through false positives and mark them as such, but until a level of AI we're nowhere near being near exists, you can't automate this process, so the only alternative is to ignore all vulnerabilities.

Complaining about this is misunderstanding the asymmetrical costs of different types of statistical error. Vulnerability scanners are sensitive by default because the cost of lots of false positives is annoyed developers who have to slow down their delivery cadence. The cost of false negatives is anything from compromising user data to losing your company to bringing down the power grid of a major city depending on what the application is.

Which of those is a higher cost? npm can't possibly know the answer to that, so it has to default to assuming security actually matters to you. If it doesn't, your local policies can be more lax, but don't expect the tool to change for you.

3 comments

>it's annoying to have to comb through false positives and mark them as such

There is no way to "mark them as such". That's half of the issue. The other half is that many people reporting these issues have not "opted into" any security tooling and don't understand its tradeoffs. They just ran `npm install`, and npm adopted default behavior of showing these warnings. For a lot of people this is their first programming environment.

>you can't automate this process, so the only alternative is to ignore all vulnerabilities.

I don't think the answer is necessarily automation. But as a package author, I'd like to be able to mark somewhere that a particular transitive vulnerabilities can't affect my users.

Or at least I'd like npm to offer a reliable way to update packages deep in the tree and override the resolved versions. Currently, there isn't such a way.

> The other half is that many people reporting these issues have not "opted into" any security tooling and don't understand its tradeoffs. ... For a lot of people this is their first programming environment.

This attitude makes me kind of uncomfortable. Like, I have taught software development to a decent number of folks, but I've always done so in a relatively isolated environment. If one is buying into web programming, I have a hard time feeling like it matters that it's their first programming environment--it is a hostile place (the web) and some understanding of that hostility is pretty high on the list, I think, of Things To Get Used To. There's definitely a tension there with "don't overwhelm a novice", but I don't necessarily think optimizing for the novice case is wise, especially when we want those novices to have their heads on a swivel, too.

> But as a package author, I'd like to be able to mark somewhere that a particular transitive vulnerabilities can't affect my users.

I definitely agree with this, though, and this is a good way to help make something like `npm audit` more intelligible and useful.

> This attitude makes me kind of uncomfortable.

I think the point though is that security warnings need to be actionable and high-signal. Experienced folks are absolutely tuning out the security warnings on npm install, because 95% of the warnings are like the examples in the post -- I know they don't affect me/my use case and there's nothing I can do about them anyway. The effect is only compounded for novices who run "npx create-react-app hello-world" and immediately see something incomprehensible about a vulnerability in react-scripts > webpack > watchpack > watchpack-chokidar2 > chokidar > glob-parent. It either discourages them from programming entirely or it teaches them to ignore security warnings.

I don't disagree with your overall point -- e.g. we should absolutely teach novices "here's what XSS is and how to avoid it" early and often. But if a dependency manager is going to surface a vulnerability alert every time I install dependencies, the alerts should be 1) high severity (to the point where I should actually stop using the package if I am unable to patch/upgrade) or 2) at least immediately actionable. The current npm audit implementation does the opposite -- 95% of the alerts are totally irrelevant to my actual security posture, and the suggested command to upgrade a vulnerable dependency is unreliable and can actually downgrade to an older, even-less-secure version (!).

Yeah, this I totally agree with. Actionable alerts are important. The idea that novices should be opted out of ecosystem security concerns generally, less so (not least because they'll create their own security problems in the process).
Frontend is however typical first place to develop in. And backend should never trust frontend anyway, making frontend actually lower risk place. You will hack yourself basically.
> Frontend is however typical first place to develop in

I think that, even today, this is a bold claim to make. Do you have something to substantiate it?

That is not the cost of false positives. The cost of false positives is ignoring it completely. I have had jobs where we just block the security scanners because they won’t listen to any feedback about why what they are scanning was intentionally setup for the purpose of testing vulnerabilities on an internal only network. Additionally at other jobs security tickets just start to get ignored because they send too many tickets that do not matter. I feel the security field likes to ignore most feedback and play holier than though. At all companies i worked at i have only had one good security team that worked with people instead of just throwing things over the wall.
I agree about the false positive problem. Boy who cried wolf and all. I've also worked with security vendors who offer to run "free" vulnerability scans for you, and to absolutely nobody's surprise, they find vulnerabilities that just happen to be the ones that they can fix, if you buy what they are selling.

Still, your example is problematic. Beware the "internal-only network". Such a thing has mostly lost meaning today, and it was never much more than a picket fence anyway. "All devices must be capable of maintaining their security policy on an un-trusted network." https://collaboration.opengroup.org/jericho/commandments_v1....

The issue was "intentionally setup for testing purposes".
What testing purposes? How long was it up? Without knowing more, that doesn't eliminate it as a problem. There are plenty of cases where something set up "just for testing" ended up being the entry point for attackers.
The alternative is hiding most of the reports by default. It's a firehose, and almost all of it is guaranteed to be useless. For example, denial of service in a devDependency shouldn't be shown unless the user specifically requests it. Denial of service shouldn't be marked "high" importance, even though occasionally it might be quite important to some user.