| Hi Jeremy, I'm currently reevaluating the CI/CD story for my company. We're a small startup with ~20 engineers. I would like to give you some feedback and questions on Azure Pipelines. We're currently using gitlab.com for code hosting and CI/CD. We're in the Silver / Premium pricing tier so have most of the premium features. Our software stack is mostly on GKE Clusters on GCP and we also run gitlab runner in a GKE cluster. We're honestly not super happy with gitlab.com due to performance, stability and UX issues - overall not super happy with the overall quality/depth of the product. That's why we're considering to make the switch to github.com and reconsider the whole CI/CD story. I evaluated about 20 CI/CD platforms, but unfortunately didn't find the perfect one. There was always at least one important feature missing or the management of the system (i.e. jenkins, concourse) would've been too much overhead for a company of our size. My preliminary conclusion is that we should probably stay with GitLab CI/CD for now (even when switching to github.com for code hosting) since it is good on most dimensions (far from perfect though) and has low management overhead. However I gave yesterday Azure Pipelines also a shot and liked many things I saw.
My impressions were like that:
Pros:
- Flexible build system / environment, can run on VMs, containers, self-hosted agents etc.
- Reasonably complex builds can be expressed in the yaml.
- Reasonable pricing
- Big plus: Very sophisticated release / deployment pipeline management. Nice sweet spot between solutions with no / limited CD support (e.g. circleci, gitlab) and complex systems like spinnaker.
- very good github integration (uses checks API) Cons:
- Big con: Self-hosted agents cannot autoscale. For comparison gitlab runner k8s executor spawns kubernetes pods on demand so the infra / cluster footprint is really small in idle times during the weekend or nights. In comparison azure pipelines seems to only work with non-ephemeral agents and one has to set a fixed statefulset count: https://github.com/Azure/helm-vsts-agent
- Big con: Release Pipelines are only configurable via UI instead of code
- No explicit docker layer caching support (vs codefresh, circleci)
- Many out-of-box integrations / features are tied to Azure Cloud, for other clouds (i.e. GCP) one has to do write custom scripts or integrations. The two biggest cons for me are really the non-autoscaling self-hosted agents and that release pipelines are only configurable via the UI. The release pipelines only-UI problem I consider a blocker for us. In other words: If you would allow configuring release pipelines in code we would probably switch to github.com + Azure pipelines. FYI I prefer expressing complex pipelines in a real and common programming language like TypeScript or Python3 (think Jenkins: Groovy, Airflow: Python, TeamCity: Kotlin Script etc.) over YAML, even though I can live with YAML for now. Is your team working on enhancements to any of that and what would be the rough timeline? Thanks! |
We're working on the design for autoscaling self-hosted agents. We call it "Bring your own subscription". Ya, it needs a better name. You can read more about our thoughts here. https://github.com/Microsoft/azure-pipelines-agent/blob/mast...
We're adding CD/release features to YAML pipelines right now. I'd like to chat about the features you want to see first since we're prioritizing.
Docker layering caching is something we're discussing. There are some trade-offs with it vs. other approaches.