| We have a CI pipeline for a containerized python app and a react app. We have a monorepo and only trigger certain jobs depending on code changes. Our CI runs through Gitlab CI on a GKE cluster, which gives us a lot of control over the parallelism and the resources allocated. Our pipeline typically takes 10-30 minutes, depending on what jobs run and where cache gets used. The longest job, at a consistent 12 minutes is our backend test job. There’s not a lot we can do to speed this up any further because a lot of the tests run agains a test db so we can’t easily run them in parallel. Perhaps if we wanted to be really clever we could use multiple test dbs. The build for our containers is usually very quick (a few minutes) unless we modify our package requirements.txt. That happens infrequently but it triggers an install step that will increase the overall time for the job to 10-12 minutes. The deploy phase is very quick. We spent a bit of time optimizing this and it came down mostly to: 1. Using cache where we can. 2. Ensuring we had enough resources allocated so that jobs were not waiting or getting slowed down by lack of available cpu. 2. Making sure that each command we run is executing optimally for performance. Some commands have flags that can speed things up, or there alternate utilities that do the same thing faster. One example of the latter is that we were using pytype as our type checker, but it often took about 15 minutes to run. We swapped it out for pyright, which takes under 5. |