Hacker News new | ask | show | jobs
by aseipp 908 days ago
In the case of the Remote Execution/Cache API used by Bazel among others[1] at least, it's a bit more detailed. There's an "ActionCache" and an actual content-addressed cache that just stores blobs ("ContentAddressableStorage"). When you run a `gcc -O2 foo.c -o foo.o` command (locally or remotely; doesn't matter), you upload an "Action" into the action cache, which basically said "This command was run. As a result it had this stderr, stdout, error code, and these input files read and output files written." The input and output files are referenced by the hash of their contents, in this case, and they get uploaded into the CAS system.

Most importantly you can look up an action in the ActionCache without actually running it, provided you have the inputs at hand. So now when another person comes by and runs the same build command, they say "Has this Action, with these inputs, been run before?" and the server can say "Yes, and the output is a file identified by hash XYZ" where XYZ is the hash of foo.o, so you can just instantly download it from the CAS.

So there are a few more moving parts to make it all work. But the system really is ultimately content-addressed, for the most part.

[1] https://github.com/bazelbuild/remote-apis/blob/main/build/ba...

1 comments

If you're only using remote caching (ie no remote execution) then all cache clients need to trust each other, because a malicious client can upload any result it wants to a given ActionCache key, and there's no way to verify the ActionCache entries are correct unless the actions are reproducible. (And verifying ActionCache entries by rerunning the actions kind of defeats the purpose of using a build cache.)

If you don't want clients to have to trust each other, then you can block ActionCache write access to the clients and add remote execution. In this setup clients upload an action to the CAS, remote executors run the action and then upload the result to the ActionCache, using the hash of the action as the key. This way malicious clients can't spoof cache results for other clients, because other clients won't ever look for the malicious action's key in the ActionCache.