[jeffbee]

I currently wait twenty minutes for CI to package a Go program that can be built on a laptop in ten seconds, because the “CI pipeline”—and every time I have to say that I choke on the words a little —involves twenty dependent steps, each of which installs Ubuntu. And this is a large, valuable company with thousands of engineers. I think having a development environment where the speed of the linker is even remotely relevant is a problem faced by a few special companies.

[parentheses]

Why not containerize the build so you can cache these repeated steps?

I'm sure you already thought of it but curious what is in your way here.

[mike_hearn]

Sounds like it already is containerized, and that's the problem? A lot of places have CI setups that don't cache things anymore for various reasons. A while back there was a new YC startup announced whose product was essentially a SaaS for building Docker containers where they cache things for you.

Another cause of this problem is when CI workers are spun up and down on demand in the cloud, so VMs are constantly being set up from scratch.

[eikenberry]

The same reasons you can't just inline everything in a program, but break things into functions, modules, etc. Complex CI systems are, in my experiences, built up from components that are reused across multiple project builds and need to be encapsulated in a way that they can be plugged into any of those. Each component is containerized and caches as much as possible at each step, but that is still a lot of overhead.

[jeffbee]

I'm just not in charge of build and release. The improvements needed seem obvious to me.