[rafaelturk]

As bad as this may sound, this is why a love Open Source, npm and the JavaScript ecosystem. It super easy to audit and check the code.

What is missing is more automated and recurrent checks in all the packages and downstream dependencies.

[qwertox]

I just can't agree with this. The problems npm has are not new, surprising ones. They are happily letting people upload malware.

https://my.diffend.io/npm/coa/2.0.3/2.0.4/

In 2021, why on earth does such a change not trigger a review before release?

[rafaelturk]

I know npm feels like wild west, but you can audit. Its quite a challenge to review imany of the C, C++, libraries out there that are just a .zip file stored in a website.

My point is that: Npm is auditable, trackable. I'm not challenging the bug itself, neither the security issue..

[Kiro]

Trigger a review where and by who?

[qwertox]

The receiving server would trigger it. The first review should be done by the owner, who should be contactable via email and authenticate via 2FA and acknowledge the modification.

Then I'd guess that Microsoft has enough information with NPM's history to train an AI. Specially the modifications made in these versions could easily trigger suspicious activity.

Did you look at the diffs?

Also, three years of inactivity and then a sudden upload should easily trigger a manual review, even if it is by automatically opening an issue with a review request on that project's GitHub page.

[zibzab]

That exists already, for example github does it automatically for you.

However, that is dependent on first finding and flagging issues, which is exactly what this post is about.

[0xdeadb00f]

This is a really odd comment. Npm, and the strange, insecure JavaScript packaging ecosystem is the reason this happened in the first place.

[ziml77]

I don't see how this is an NPM issue at all. If the attacker made a Linux version and stuck it inside a Makefile, plenty of devs and build servers would end up executing the malicious code.

[jacquesm]

Yes, but good luck getting kernel.org to distribute that. They're not perfect (nothing is, and kernel.org has been compromised in the past, though quite long ago, and even then it did not lead to compromise of the Linux kernel sources), but to put the Linux kernel maintainers on the same level as the people maintaining the NPM distributions is a bit much.

[nyanpasu64]

On npm, the average application uses hundreds (or thousands?) of dependencies, none of which are audited by the application author. On Linux, ffmpeg uses dozens or hundreds of dependencies, but they're each trusted and audited by the Linux distro packaging the dependency packages, who would hopefully catch malware shipped in a Makefile. And whatever vendored libraries a C/C++ app bundles (so you can build it on distributions that don't ship the library or a new-enough version) are usually self-contained rather than having transitive dependencies. As a result, a C++ app's author count grows linearly with the number of dependencies you bring on (and you generally evaluate each author's trustworthiness, and hopefully skim over the library and build scripts as well), rather than polynomially in the dependency tree's branching factor and exponentially in its depth (you generally don't evaluate the trustworthiness of your transitive dependencies' authors, since there are so many and they're not shown to you when you add a dependency).

Additionally, an app's vendored dependencies don't automatically update until you choose to update them. And when you update a vendored dependency, you have a chance of finding malware when you review the diff before committing the update. The downside is that vendored dependencies introduce friction in updating your dependencies (I've experimented with extracting tarballs, git subtree, and git subrepo, and generally avoid submodules, and they're all somewhat awkward), and don't automatically receive patch/security updates like distribution libraries do.

I'm more familiar with crates.io and PyPI than npm, but I think they all work similarly (unvetted package repository allowing libraries to depend on other libraries, and transitive dependencies get pulled in when you install the library, and can be updated without the library author's knowledge). The primary difference IMO is the degree that packages use transitive dependencies vs. building/copying the subset of code they need (the average Python app uses dozens of dependencies, Rust apps use hundreds, Node apps use thousands).

My wish is that for each language, a team of Linux distribution maintainers or community members would "sign off on" a set of trusted packages, each of which depends on only trusted packages. And when a trusted package gets updated, the new version isn't marked as trusted until the maintainer verifies the diff isn't malicious. Then the language package manager would add a mode where apps or libraries can choose to only depend on trusted packages (which only depend on trusted packages, etc.).

(In the Rust world, where the compiler and package manager are developed by the same non-commercial organization, I hope the Rust organization will create the team handling "trusted packages"; sadly they seem opposed to it because they consider it favoritism, but IMO it's a good thing for Arch to package wget and curl, but not "joe's 1337 haxxor tool". In the Node world, V8 and node.js and npm are different groups/companies I think, so I don't know which organization is community-led and neutral enough to make a trustworthy host for this effort.)

Crev takes the "web of trust" approach, where everyone can choose who to trust. However, it doesn't ship a "deny untrusted dependencies" mode as part of Cargo. And as with PGP webs of trust, I haven't met any library developers in person, don't know which developers sign off on packages, and would prefer having semi/centralized institutions (like Linux distributions) who stake their reputation on the non-maliciousness of packages they ship, and allow different people to agree on which packages to trust. Having institutions reduces the responsibility placed on the individual app developer. Right now, you need to trust an distro to take responsibility for an OS, and a language team to take responsibility for a compiler. With crev, you'd have to also search for trustworthy people who've reviewed enough package that you can feasibly build apps using only those dependencies, instead of having the distro or language team provide that service out of the box.

Maybe I'm just too lazy to find trustworthy people. But currently there are not enough Crev reviews at all. Searching through https://web.crev.dev/rust-reviews/crates/, there's no mention of gtk and pipewire, and rusqlite and sqlite are reviewed but not libsqlite3-sys. And if I were to start writing reviews, would people trust me?

My hope is that Linux distributions will enforce a policy of only packaging applications where all dependencies are trusted packages (on top of reviewing the app's own codebase as usual). As difficult as this may sound, this is already the norm for C/C++ applications, where it's easy to find either distribution-packaged or self-contained libraries, and there are organized teams of maintainers selecting trustworthy packages, auditing and packaging libraries based on user demand, and telling audio libraries that require 70 packages to bind to PipeWire to clean up their act. Unfortunately, in languages where package managers and communities have embraced unvetted transitive dependencies, it's difficult to even find libraries with small or trusted dependency trees. This normalization of deviance has seriously harmed our ability to understand and/or trust the code making up our apps, and it will take changes in our package managers and community norms to restore trust.

[jacquesm]

This eloquently describes the root cause of the problem.

[rafaelturk]

I know npm feels like wild west, but you can audit. Its quite a challenge to review many of the C, C++, libraries out there that are just a .zip file stored in a website.

My point is that: Npm is auditable, trackable.

I'm not challenging the bug itself, neither the security issue..

[Kiro]

What makes npm more insecure than other packaging systems?

[jacques_chester]

I would say that it's not that different from others I've seen, just more visible because of the size and activity of the repository.

One thing NPM does (and I believe Python too) is to allow install scripts -- this has been a reliable vector for attackers to steal credentials. Not every package repository system has that.

[avereveard]

authors cannot revoke their compromised keys to immediately halt all distribution, and you don't have any process to verify package<->author ownership beyond the upload secrets.

[rafaelturk]

Question for all folks downvoting this. How do you perform same audits with C++ libraries? CocoaPods? Pip?