[jchw]

To be honest with you, I don't know what you are suggesting. Piper is a more scalable Perforce clone. That's cool and all, but I don't see how it solves the problem that hosts can be unreliable and that data fetched from the internet can be changed later. Doesn't seem to me like there's anything a VCS itself can do to change this.

You maybe could invent a fully decentralized peer-to-peer VCS hosting system that uses Blockchain(TM) technology to ensure integrity, or something. But... You'd probably still want a cache, perhaps even moreso than before, for performance reasons, even if you really truly did solve the availability and integrity problems.

[SkiFire13]

> and that data fetched from the internet can be changed later

So do you want the VCS to be the source of truth or not?

[jchw]

First of all, can you just simply not speak in gotchas? I don't even like engaging with this sort of thing. It's a valid question to ask how VCS is the source of truth in this case, but the way you're responding suggests you're not interested in discussing it productively, just a shit-flinging where we argue semantics about what it means for something to be a source of truth, without regards to what the actual important implications are supposed to be. I'm going to try to interpret this in good faith but I feel like I'm probably wasting my time doing so.

Yes, the VCS is still the source of truth, ultimately nothing in the cache or sumdb exists without having existed in the VCS source first. The sumdb is an integrity mechanism bolted on top, and then the module proxy cache is an availability mechanism bolted on top.

Consider the following: let's say Kubernetes depends on some library you have Git access to. So you, being evil, rewrite the tag Kubernetes currently uses to contain malware. There are two possible scenarios:

- By default: the user is using a module cache, so changing a tag that already existed is no different than a Git host being pulled offline from the perspective of the user. The cache continues to serve the original source code, no malware delivered.

- The decentralized case: a user disables both sumdb and the module cache. Guess what? The pull fails. The pull fails because the project has a go.sum file and your tag doesn't match it.

Let's say someone compromised Google's sumdb or module proxy cache. Same basic thing: the go get will fail, because of the local integrity checks.

See, it really changes nothing. The module proxy cache essentially just ensures that a module source code that was once available remains available indefinitely so that you can continue to build old software exactly and securely.

The only place where behavior ever changes meaningfully (as in, not just "available or unavailable") is when go getting a new package or package update. If you have GOPROXY=direct, the poisoned fetch will fail because of global sumdb. If you disable that too, it may still fail because of local sumdb. But if not, you may get someone to pull your mutated tag. (And then the resulting go.sum would fail for other people.)

If you knew for absolute certain that nobody had ever pulled the package before, then theoretically mutating an existing tag would be safe. However, I call shenanigans on this: if you knew for sure nobody had ever pulled it, how in the world did it end up in the sumdb and module proxy cache? Granted, it may have been because of your own CI or something, but still, it stands to reason that this mechanism is introducing a highly desirable property. If a given tag was available once with a given source code I would want to be able to pull it again later and know it isn't tampered with.

In general, I have never seen it interpreted that the possibility of a stale cache changes what the canonical source of truth is. I've also never seen anyone argue that a lockfile changes the canonical source of truth. The Go sumdb being a global integrity mechanism does make it fairly unique, but I don't see it as being categorically different than the integrity checks from lockfiles, it's just that but across multiple users.

Even if you don't like Go's implementation particularly, I still think it is highly worth copying. You can always change a few things. Like for example, you could make module fetch operations not backed by a local checksum skip the global module proxy cache as a safeguard against the proxy itself, only allowing it to be trusted once you already have evidence that the sumdb and canonical source currently agree by virtue of already having a sum.

Sorry I have dropped another text wall, I don't have time to make it shorter. It will happen again.

(I realize that I have ranted in two directions here, but my intention was to 1. Argue that the properties introduced by the design are actually important and desirable, 2. Then argue semantics about whether it changes what one would consider to be the canonical source of truth, second mainly because I consider this to be unimportant. I mean it could be discussed further but, there really should be a point to it other than just semantics.)