[flicaflow]

Nice to have some development in this space. However I would be very interested in some discussion why this approach should be superior to goBuildModule, which is also mentioned in the article. goBuildModule works well for us and it is even arguably simpler to use, you just have to update the vendor hash whenever you updated the dependencies. The only real downside for us is that it doesn't handle private dependencies well.

[pxc]

The announcement post for gomod2nix has some additional info on the problems with buildGoModule's approach to vendorization:

> The buildGoModule package is designed around fixed-output derivations, which means that a single derivation is created where all the dependencies of the package you want to build are wrapped, and only a single hash of the derivation is specified. It fetches all dependencies in the fixed output, creating a vendor directory which is used for the build.

> This has several issues, most notably there is no sharing of dependencies between packages that depend on the same Go module.

> The other notable issue is that it forces developers to remember editing the vendorSha256 attribute separately from the already existing hash/sha256 attribute on the derivation. Forgetting to do so can not only lead to incorrect builds but also be frustrating when working with larger packages that takes a long time to build, and only very late in the build notice that something was broken so you have to start over from scratch.

> Because of the lack of hash granularity the build needs to clone every dependency every time the vendorSha256 is invalidated, and cannot use the cache from previous builds.

https://www.tweag.io/blog/2021-03-04-gomod2nix/

So you get greater granularity here, and thus also greater cache reuse between builds.

[lewo]

Another advantage of the gomod2nix approach concerns long term build reproducibility [1]. Having each source dependency in its own derivation would make easier source code mirroring. Since Nix knows the hash of each sources, we could imagine to teach Nix how to get the source code from some other places than GitHub. For instance, Nix could use the hash of the dependency to query Software Heritage [2].

Note there is also an opened issue to restrict Nix fixed output derivation which would make impossible the buildGoModule "hack".

[1] https://github.com/NixOS/nixpkgs/issues/84826

[2] https://www.tweag.io/blog/2020-06-18-software-heritage/

[pxc]

That's actually really important. Sometimes upstream sources do disappear, and the caches we have don't help very much, or only help for the exact collection used in a specific package, when they're one big chunk like with buildGoModule.

I hope that FODs in Nix itself stay flexible, though, even if a more limited form becomes preferable in Nixpkgs. New fetchers being possible to implement in Nixpkgs rather than in Nix itself seems really valuable, and clunky uses of FODs to wrap upstream vendorization are still an important fallback option for wrapping new or ill-behaved upstream package managers, until better implementations are worked out or become possible.

[maherbeg]

Is there any reason this isn't just merged into the buildGoModule package? It seems like a net positive?

[ghuntley]

See https://code.tvl.fyi/tree/nix/buildGo as an alternative.

"Most language-specific Nix tooling outsources the build to existing language-specific build tooling, which essentially means that Nix ends up being a wrapper around all sorts of external build systems.

However, systems like Bazel take an alternative approach in which the compiler is invoked directly and the composition of programs and libraries stays within a single homogeneous build system.

Users don't need to learn per-language build systems and especially for companies with large monorepo-setups (like Google) this has huge productivity impact.

This project is an attempt to prove that Nix can be used in a similar style to build software directly, rather than shelling out to other build systems. "

[xena]

The main difference is that it's easier to track what changed in the vendor closure rather than just waiting for the build to fail and updating the hash with the result that Nix gives you.

[flicaflow]

but how does it react when go.mod and the toml get out of sync? For us this will just break on CI and requires then a commit with the updated vendorSha to be fixed. I would expect the same workflow from this approach?

[tadfisher]

The only reason they would get out of sync is if you updated dependency versions without also running gomod2nix. The benefit here is that you can do that without the intermediate step of first building the project and updating the hash after the build fails.