Hacker News new | ask | show | jobs
by jchw 23 days ago
Sure, you can tell your other package managers to pull a module from VCS, usually with some limitations and negative side effects. The Go module syste is not the same model. You can, for example, run an NPM proxy that caches locally, but what you're missing is that with Go, the proxy part is the centralized part. There is no NPM. There is no crates.io, no PyPI, no RubyGems, no Packigist, no PECL, no CPAN. The proxy's entire job is just to stand in for requests that would be made to individual VCSes using their original source repositories. The sumdb's entire job is to ensure that whether proxied or not, the source code you get isn't tampered with. The source of truth remains decentralized, in the individual VCSes.

With NPM, uv, composer, Cargo, etc. you can indeed choose to make your dependencies pull from VCS, but:

- Not having universal module proxies with caching means you will run into the usual problems. Repos can disappear, history can be rewritten, hosts can go down. Even if you invented an equivalent caching service, without making it an ecosystem default it won't achieve the same effectiveness as Go because less would be in the cache.

- Because it isn't an ecosystem standard, dependencies you pull would then still reference dependencies from centralized repos anyways, more or less defeating the point. You'd have to go and override every recursive dependency to really make it decentralized.

(And again, it usually comes with other downsides depending on the specific ecosystem. Performance is a big one, the module cache definitely helps make fetching dependencies in Go faster.)

Not having centralized package repos comes with its ups and downs, but the lack of one existing means there are a limited number of interesting attacks you can really pull in the Go ecosystem and they are heavily tamper-evident. You can try to poison the module cache with a given revision for a dependency with malicious code then overwrite it with an inoccuous commit, but this will probably be detected when someone (possibly in Nixpkgs or another packaging system) runs GOPROXY=direct and the sumdb doesn't line up. So the least noticeable thing you can really do is just leave your malicious code in the repo. Go also has the kind of nice property that fetching and building modules is supposed to be safe with untrusted code. That's pretty good, it would be hard to do better than that.

It also means there really isn't any central namespace to poison. You can still trick someone into downloading and using malicious software but you're mostly limited to doing it the old fashioned way. Again, hard to do much better than that.

While everyone has been losing their minds over supply chain issues, the Go ecosystem has been concerned but less so. It's still a huge risk still, dependencies can get compromised still, but attackers have no real central target to go after and get a bunch of damage at once. Nothing realistical, anyway. So, it's back to just trying to pwn individual GitHub repos.

2 comments

> There is no NPM.

any package manager can be redirected towards particular sources. What is unique about go? I sense, though this is not articulated through your communication, that the source of the package is directly specified in the source code requiring the package. Is that what you're trying to say—source pinning is evident in the source?

What's unique about Go is the lack of a centralized package repo. You are saying "I could point NPM to another registry". I am saying, Go doesn't have a registry and you can't really point it to one. I hope it is apparent that these are two extremely different statements. The fact that it is pinning source to source is just the approach that they take to get there.

And again, if you just bolted a solution onto the side of an existing ecosystem it wouldn't accomplish as much. "Technically this could be done elsewhere" is true, but I hope you know I know that. The point is that Go deployed a mostly decentralized package management solution to the entire ecosystem as the primary and essentially only way to manage packages with Go. You actually could approximate what Go does using NPM, sans the caching module proxy part, using VCS URLs as dependencies, but the "approximate" and "sans caching module proxy part" is doing some heavy lifting.

There isn't really anything that does what Go does exactly. The closest example to me is OCI images. Like Go modules, their identity is essentially a sort of URI. However, it isn't a language package manager and has fairly different challenges, and doesn't really seem to need the caching proxy stuff due to the relatively few registries (and running it would be very expensive due to image sizes.) So I stand by with my belief that what Go does here is pretty unique.

> The source of truth remains decentralized, in the individual VCSes.

That's not entirely true, if the VCS's tag changes the proxy might not pick it up.

Changed tags are not allowed in Go (though you can work around it if you really want to boil yourself). All references are hashed so any attempt to alter a published package version will get blocked.

Go has a decentralized append only auditable hashsum db protocol.