Hacker News new | ask | show | jobs
by lrvick 79 days ago
An owner being compromised is absolutely survivable on a responsibly run FOSS project with proper commit/review/push signing.

This and every other recent supply chain attack was completely preventable.

So much so I am very comfortable victim blaming at this point.

This is absolutely on the Axios team.

Go setup some smartcards for signing git push/commit and publish those keys widely, and mandate signed merge commits so nothing lands on main without two maintainer sigs, and no more single points of failure.

4 comments

Did you investigate the maintainer compromise and publication path? The malicious version was never committed or pushed via git. The maintainer signs his commits, and v1 releases were using OIDC and provenance attestations. The malicious package versions were published locally using the npm cli after the maintainer's machine was compromised via a RAT; there's no way for package maintainers to disable/forbid local publication on npmjs.

It seems the Axios team was largely practicing what you're preaching. To the extent they aren't: it still wouldn't have prevented this compromise.

I can not find a single signed recent commit on the axios repo. It is totally yolo mode. Those "signed by github" signatures are meaningless. I stand by my comment in full.

One must sign commits -universally- and -also- sign reviews/merges (multi-party) and then -also- do multi party signing on releases. Doing only one step of basic supply chain security unfortunately buys you about as much defense as locking only a single door.

I do however certainly assign significant blame to the NPM team though for repeatedly refusing optional package signing support so packages with signing enabled can be refused at the server and client if unsigned by a quorum of pinned keys, but even aside from that if packages were signed manually then canary tools could have detected this immediately.

What you sign or don't sign in your Git repo doesn't matter because NPM doesn't publish from a Git repo. Signing commits is still useful for your contributors and downstream forks but it won't have any effect on the users who use your package via NPM.

I think NPM is fully to blame here. Packages that exceed a certain level of popularity should require signing/strong 2FA. They should implement more schemes that publishers can optionally enable, like requiring mandatory sign-off from more than 1 maintainer before the package is available to download.

Then on the package page it should say: "[Warning] Weak publishing protection" or "[Checkmark] This package requires sign-off from accountA and accountB to publish".

2FA was mandated by npm

they had 2FA, but likely software TOTP (so it was either autofilled via 1password (or similar), or they were able to steal the seed)

at this point I think publishing an npm app and asking people to scan a QR with it is the easiest way (so people don't end up with 1 actual factor)

What they need to mandate is hardware anchored passkeys/fido2/webauthn for both auth and package signing, with the -option- to sign with PGP for those that have well trusted PGP keys.

They won't do this, I have talked to them plenty of times about it. But, if they did, the supply chain attacks would almost entirely stop.

Don't need to require hardware 2fa tokens. Just a mobile app would be sufficient. Publish to a staging area then require confirmation on mobile to make it go live. Maybe include a diff of changes files for good measure.
So you think the answer is replacing a requirement for a 6-digit 2FA code that can be typed into the npm publishing CLI with a requirement for a device that has a camera that can scan a QR code and then... what? What does the QR code do on the device? How does the npm CLI present the QR code?
Simply supporting passkeys gives people domain locked login via qr/phone, or any fido2 usb device. No more keyboard entry required for login other than username, which means phishing is off the table. Standards are great if we can get anyone to use them.
Like I said. One must sign commits -universally- and -also- sign reviews/merges (multi-party) and then -also- do multi party signing on releases. The code in the release must match the code from git, or no publish.

Until NPM can enforce those basic checks though, you have to roll your own CI to do it yourself, but large well funded widely used projects have an obligation to do the basics to protect their users, and their own reputations, from impersonation.

I agree, I just think it's pointless to discuss Axios' commit-signing practices or lack thereof when NPM doesn't support any of it. It seems like axios was already using Trusted Publishing [1] and it still didn't get caught.

You said that you "also" blame NPM, but they're the only party who should get any blame until they get their shit together.

[1] https://github.com/axios/axios/issues/10636#issuecomment-418...

It wasn’t done through git. It was a direct npm publish from the compromised machine. If you read further down in the comments (https://github.com/axios/axios/issues/10636#issuecomment-418...), it seems difficult to pick the right npm settings to prevent this attack.

If I understand it correctly, your suggestions wouldn’t have prevented it, which is evidence that this is not as trivially fixable as you believe it is.

To prevent supply chain attacks you need multi party cryptographic attestation at every layer, which is pretty straight forward, but you are correct, NPM and GitHub controls absolutely will not save you. Microsoft insists their centralized approach can work, but we have plenty of evidence it does not.

Operate under the assumption all accounts will be taken over because centralized corporate auth systems are fundamentally vulnerable.

This is how you actually fix it:

1. Every commit must be signed by a maintainer key listed in the MAINTAINERS file or similar

2. Every review/merge must be signed by a -second- maintainer key

3. Every artifact must be build deterministically and be signed by multiple maintainers.

4. Have only one online npm publish key maintained in a deterministic and remotely attestable enclave that validates multiple valid maintainer signatures

5. Automatically sound the alarm if an NPM release is pushed any other way, and automatically revoke it.

And for 5 there should be help on the NPM end to make it so that the alarms can fire before the new update is actually revealed to the public. There could be a short staging time where it could be revoked before any harm has been done. During this staging time NPM should also scan the package through a malware scanner before allowing it to go public.
I agree that would be nice, but NPM absolutely will not do any basic supply chain integrity work. They are actively opposed to it citing concerns that it might turn off lower skill developers that would be too annoyed by tapping a yubikey to sign releases or code. I have talked to them enough times over the years to have completely given up here.

Whats even more stupid is they actually started mandating 2FA for high risk packages, and FIDO2 supports being used to actually sign artifacts, but they instead simply use it for auth, and let releases stay unsigned. Even the developers they insisted hold cryptographic signing keys, they insist on only throw-away signatures for auth, but not using them for artifact signing to prevent impersonation. It is golf clap level stupid.

Consider them a CDN that wants to analyze your code for AI training for their employer and nothing more. Any security controls that might restrict the flow of publishing even a little bit will be rejected.

The "nothing gets on main without two signatures" rule would not have prevented the xz story, where a comaintainer was able to smuggle malicious code past the review as "binary data for new tests" and, effectively, get it signed.
This only works up to a point. Some human needs some way of changing the publication setup in case something goes wrong or changes. What you're asking is blowing a proverbial e-fuse once the setup is known to be working. This is software, shit will go wrong at some point and you need a way to make changes.
Of course, which is why all the (decent) tooling for this is provider agnostic, and provides documentation for multi-party-sharded backups so a quorum of maintainers can always re-assemble the key by hand for any reason if needed.