[mgaunard]

I built my own build system that does something similar.

I've set it up at work with two S3 buckets: trusted and untrusted. CI/CD read/write from trusted only. Developers read/write from untrusted, and read-only from trusted.

[sgammon]

We decided to back our main cache with in-memory storage for spicier performance. I’m curious how well S3 has worked for you here? Is it fast enough?

Or, maybe the blobs you’re dealing with are on the bigger end? That would also make sense

[mgaunard]

Each object file (.o) has a unique hash and is stored as thehash.o.

It's certainly much faster to download the .o than it is to build it. Once it's downloaded it stays on the local filesystem until it's garbage-collected.

[rurban]

> It's certainly much faster to download the .o than it is to build it.

That depends on your network latency and throughput, plus the performance of your compiler.

With fast compilers and less bloat (not C++ nor Rust) such a distcc is certainly slower.

[sgammon]

Hm, interesting. Our free tier is planned to be this plus R2, so I’m happy to hear S3-style data exchange is working for people. Thanks for sharing

[mgaunard]

The whole point of S3 is that it is inexpensive. You don't want to pay premium money for terabytes of data that are usually invalidated everyone someone makes a significant change.

[sgammon]

Our blobs end up quite a bit smaller than that, and I can see using S3 for many build cache problems. In our case in-memory worked better with overflow to disk for large or infrequently used objects.

R2 is S3 to me, btw, or at least it’s the same API. Object storage as a model is really what I’m asking about. I’m genuinely glad to hear S3 was enough to be impactful.

[mgaunard]

I mostly saw R2 as a faster alternative to S3 with a compatible API.

But it looks like it could be competitive pricing-wise as well, especially if you need to access the same data across regions.