Hacker News new | ask | show | jobs
by ilmiont 1878 days ago
I don't see how anyone can give you useful information without knowing more about the pipeline and the projects, and as everyone's pipelines/projects are going to work differently (I do web dev work, so pipelines are relatively simple, I can imagine that a game dev team creating Windows/Mac/Linux builds might have multi-hour pipelines though).

Anyway as the question is "How Long Is Your CI Process", here we go!

I have two main types of pipelines, both running on a self-hosted GitLab instance which runs on an 8th-gen i3 Intel NUC. No project is particularly massive.

1. PHP Projects. Run PHPStan + unit tests on each branch. Most projects take 1-5 mins. On master, run PHPStan + unit tests, build a Docker image, and use Helm to deploy to managed Kubernetes on DigitalOcean. This takes 5-10 mins.

2. React Projects, again not massively huge, but sizable. Biggest time is to run ESLint on every branch. About 5 mins (due to very poor caching which I keep meaning to fix). On master, run ESLint, create a Docker image, and deploy to managed Kubernetes. 5-10 mins.

There are opportunities to improve this by fixing/optimising caching. Overall I'm reasonably happy with the pipeline performance. I'm also sure that upgrading the hardware would make a big difference, probably more so than fixing the caching; an i3 isn't really ideal but this machine does well overall for my small team.

1 comments

Computing is cheap these days. If somewhere has a multi-hour build, that is a noxious build system smell. Run away. Or make sure the role involves you getting paid to optimize it. If a build takes 2 hours, and you work 8 hours days, it means you get four tries in one day to get it right? Four!

That is no way to do modern computing. Things were worse back in the day (we compiled uphill, both ways), but we're not in those days any more. Buy bigger & faster servers with more RAM until the problem goes away. It won't necessarily be cheap. But if the company is too stingy, and would prefer to be pennywise and pound foolish (saving "pennies" on a server vs developer time to sit there waiting for "compiles"), you don't want to work there.

What drives me a bit nuts is when dynamically evaluated languages (which theoretically punt their compilation to lazy on-demand runtime compilation/evaluation) become monstrosities in the build cycle complete with painfully slow compilation and package resolution.