[jchw]

It’s very subtle, but there are some important differences. For example, lockfiles are not recursive in NPM: the NPM package (usually?) does not contain the lockfile and does not adhere to it when installed as a dependency. It will pick the newest version of dependencies that matches the spec in package.json.

Go mod files are used recursively, and rather than try to pick the newest possible version, it will go with the oldest version.

This avoids the node-ipc issue entirely, at least until you update the go.mod.

[steveklabnik]

This really depends on the specific package manager: if you're building an application in Rust, its lockfile will contain the full tree of dependencies, locked to a specific version.

[saghm]

I might be misunderstanding GP, but I think what they're saying is that when package A depends on package B, building package A will use B's lockfile. Assuming that's the case, I think this is generally not how Rust does things, as by default Cargo.lock is explicitly listed in the .gitignore when making a library crate, although there's nothing stopping anyone from just removing that line. I think I remember reading in documentation somewhere that checking in Cargo.lock for libraries is discouraged (hence the policy), but I don't recall exactly where since it's been so long. (That being said, there's a pretty decent chance you were the one who wrote that documentation, so maybe you might remember!)

[steveklabnik]

> I think this is generally not how Rust does things

That is correct.

> as by default Cargo.lock is explicitly listed in the .gitignore when making a library crate

Even if it is included in the contents of the package, Cargo will not use it for the purpose of resolution.

The "don't check it in" thing is related, but not because it will be used if it's included. It's because of the opposite; that way new people who download your package to hack on it will get their own, possibly different Cargo.lock, so you end up testing more versions naturally. Some people dislike this recommendation and include theirs in the package, but that never affects resolution behavior.

[saghm]

Ah, good to know! I'm glad I asked

[jchw]

Yes, this is true; I think the article is definitely making the distinction vs NPM and not Rust.

[rjmorris]

The article seemed to go out of its way not to mention any specific package manager or ecosystem. So I think comparing to Rust is completely reasonable.

[jchw]

It didn’t mention one by name, but Rust hasn’t been subject to any widely publicized supply chain attacks. They do, however, mention left-pad by name. I think it can be implied that they really did just mean npm.

[munificent]

They definitely should have said that then. Simply referring to "lockfiles" paints with a very broad brush that includes a number of package managers that don't have the problems that NPM does.

[typical182]

The default way Go handles go.mod is fairly different than the default way Cargo handles Cargo.lock files, including for example with libraries.

Also, when the blog says:

> Moreover, when a dependency is added with go get, its transitive dependencies are added at the version specified in the dependency’s go.mod file, not at their latest versions, thanks to Minimal version selection.

I believe that is significantly different than default Cargo behavior and for example default 'pub' behavior for Flutter (though I know approximately nothing about Flutter package management beyond a cursory search just now ;-)

To my knowledge, both Cargo and Flutter 'pub' prefer the most recent / highest allowed version by default when asked to solve constraints, whereas Go does not.

Cargo: [1]

> When multiple packages specify a dependency for a common package, the resolver attempts to ensure that they use the same version of that common package, as long as they are within a SemVer compatibility range. It also attempts to use the greatest version currently available within that compatibility range.

Flutter 'pub': [2]

> For each package in the graph, pub looks at everything that depends on it. It gathers together all of their version constraints and tries to simultaneously solve them. (Basically, it intersects their ranges.) Then it looks at the actual versions that have been released for that package and selects the best (most recent) one that meets all of those constraints.

[1]: https://doc.rust-lang.org/cargo/reference/resolver.html

[2]: https://dart.dev/tools/pub/versioning#constraint-solving

[jchw]

I believe Cargo does still have less strictness over dep versions than Go modules, since it will never use a module newer than the one specified in any go.mod file. Lockfiles are generally not honored recursively, and I don’t think Cargo is different here? Hope I’m not spreading misinformation, though I couldn’t find any docs with a cursory glance.

I don’t want to make assertions that I’m less sure of, but I think NPM and Cargo are actually more similar than different here. They both specify exact versions in lock files, for all nested dependencies, but don’t honor the lock files present inside dependencies, instead calculating the nested deps from the constraints.

[fiddlerwoaroof]

This is also true of npm and yarn, as far as I can tell: package-lock.json and yarn.lock contain the exact version of every transitive dependency.

[steveklabnik]

I always forget the exact semantics, but the parent's description of them as "recursive" is not the same as Cargo; Cargo determines the full tree and writes out its own lockfile, if dependencies happen to have a Cargo.lock inside the package, it's ignored, not used.

[lamontcg]

How does go deal with the diamond dependency issue when transitively you've a dep on the same package via two different paths?

[jchw]

As far as I understand it... the import path that you use to import a package acts as its identity, and only one version of any given package will be installed. The way that it will determine this is by choosing the lowest version specified in any package that depends on a given package. Major versions of packages are required to have different import paths with Go modules, so when depending on two different major versions of the same package, they are treated effectively as their own package.

[typical182]

I think you might have a small typo here:

> by choosing the lowest version specified in any package that depends on a given package

It picks the highest version specified in any of the requirements. (That's the minimal version that simultaneously satisfies each individual requirement, where each individual requirement is saying "I require vX.Y.Z or higher". So if A requires Foo v1.2.3 and B requires Foo v1.2.4, v1.2.4 is selected as the minimal version that satisfies both A and B, and that's true even if v1.2.5 exists).

[jchw]

Yep, I regrettably described the system wrong. It’s the highest version specified, and thus minimum version possible.

[typical182]

That's a nice & concise way to describe it.

[lamontcg]

Minimalistic Version Selection might have been a better term for it.

That did confuse me a lot. Selecting the maximum makes a lot more sense, since that gets SemVer assumptions correct.

[lamontcg]

Okay but if I depend on A and A depends on C and has it pinned at 1.5, but I also depend on B and B has C pinned on 1.8, then I get 1.5? But what happens if C is on 1.8 because it doesn't work with 1.5 because an API it needs doesn't exist in 1.5?

Are we not talking about the transitively pinned dependencies in the "lock" section, or are we talking about logical constraints?

Logical constraints would make more sense, but if the constraints on C across various transitive deps are > 1.5 and > 1.8, and 1.9 or 1.10 exists, I probably want the last version of those.

[typical182]

> Okay but if I depend on A and A depends on C and has it pinned at 1.5, but I also depend on B and B has C pinned on 1.8, then I get 1.5?

No, you end up with the highest explicitly required version. So 1.8 in that scenario, if I followed. (Requiring 1.5 is declaring support for "1.5 or higher". Requiring 1.8 is declaring support for "1.8 or higher". 1.8 satisfies both of those requirements).

> if the constraints on C across various transitive deps are > 1.5 and > 1.8, and 1.9 or 1.10 exists, I probably want the last version of those.

By default, you get 1.8 (for reasons outlined upthread and in the blog post & related links), but you have the option of getting the latest version of C at any time of your choosing (e.g., 'go get C@latest', or 'go get -u ./...' to get latest versions of all dependencies, and so on).

Also, you are using the word "pin". The way it works is that the top-level module in a build has the option to force a particular version of any direct or indirect dependency, but intermediate modules in a build cannot. So as the author of the top-level module, you could force a version of C if you needed to, but for example your dependency B cannot "pin" C in your build.

[coder543]

I'm fairly sure Go Modules does not support what you’re describing. It specifically avoided having a SAT solver (or something similar), unlike most package managers. You specify a minimum version, and that’s it. 1.8 would be selected because it is the highest minimum version out of the options 1.5 and 1.8 that the dependencies require. Unless you edit your go.mod file to require an even higher version, which is an option. Alternatively, you can always replace that transitive dependency with your own fork that fixes the problems, using a “replace” directive in your go.mod file.

If your dependencies are as broken as you’re describing, you’re in for a world of hurt no matter the solution. I also can't remember ever encountering that situation.

[lamontcg]

Well after 12 years of using ruby and bundler and maintaining a bundler-ish depsolver at work, and playing around a bit with cargo, I can say that it is becoming clearer as to why I don't grok go modules at all.

The lack of a depsolver is a curious choice...

I don't think my example was remotely "broken" at all, that's just another day doing software development.

[coder543]

> I don't think my example was remotely "broken" at all, that's just another day doing software development.

It's not the norm for me. If this is what you consider to be the norm, then this kind of statement doesn't make me feel any better about Ruby.

I will say that Bundler is one of the better package managers, but the existence of the constraint solver doesn't fix this problem -- Bundler doesn't allow you to have multiple versions of a single dependency. The problem is fundamentally the dependency not maintaining its compatibility guarantees, which I would definitely call "broken". Sometimes breakage is unavoidable, like with security fixes that you want to be available even to users of existing SemVer versions, but it should not be a common situation.

[woojoo666]

If I'm understanding you properly, recursive lockfiles means that if I depend on some chain of dependencies A->B->C->D->E, and E has a security vulnerability that they patch in a new version, I have to wait for A B C D and E to all update their lockfiles before the security vulnerability will be patched on my system?

[typical182]

That's not correct. You can unilaterally decide to update the version of E without waiting for anyone. Alternatively, if only C for example decides to update their required version of E, you would get that version of E if you updated your version of C (directly or indirectly), without needing to directly do anything with E yourself.

Slightly longer explanation of the mechanics here: https://news.ycombinator.com/item?id=30871730

The best complete explanation is probably here: https://research.swtch.com/vgo-principles

[woojoo666]

But wouldn't that have the same issue then? Developers decide to update their dependencies to patch any security vulnerabilities, and wind up adding installing node-ipc's malicious update

[hashhar]

The difference is that it's an explicit choice instead of other package managers who'd happily install latest compromised versions of packages by default.

[woojoo666]

But if its a manual explicit choice, that adds more friction to these patches, and many developers may not update them at all. It's a trade off

[morelisp]

A manual choice can be easily automated; an automatic choice can be difficult-to-impossible to de-automate.