| I've used GitHub Actions a little in the past, also lots more of GitLab CI (I liked their pipeline description format a little bit more), some Jenkins and recently I've settled on using Drone CI for my personal projects (disclaimer: has community and enterprise editions, the latter of which is commercial): https://www.drone.io/ Personally, it's pretty interesting to see how different tools in the space handle things, even without getting into where and how your pipelines run. It's nice to see the increasing move towards defining everything in "code", regardless of whether it's a DSL or a turing-complete language - I'll take a versioned Jenkinsfile over messing about in the UI most days. > I've long wished to be just writing (ideally) Python to define and run CI/CD pipelines. YAML is simply hell, `if` keys in such data description languages are cruel jokes. I'd say that YAML is passable on its own, but gets more and more inadequate, the harder the things you're trying to do get. Thankfully, I've been able to keep most of my CI pipelines relatively simple nowadays, along the lines of: - initialize some variables that don't come out of the box from the CI solution
- parse or process any files that are needed for the build/action, such as attaching metadata to project description
- do the build (nowadays typically just building an OCI container), or whatever else the pipeline needs to do (since you can do more than just build applications)
- save any build artefacts, push containers, do logging or whatever else is needed
Even navigating between build steps is mostly taken care of by the DAG (directed acyclic graph) functionality and choosing whether a build needs to be done is typically done declaratively in the step description (though I haven't found any solution that does this well, e.g. complex conditions).That said, there's basically nothing preventing me or anyone else from including a Python script, or a Go program, or even Bash scripts (if you don't want to think about getting an environment where most of the other languages are available, in lieu of the footguns of Bash) and just running those. Then, control flow, looping, using additional libraries or tools suddenly becomes more easy. > Round-trip times for GitHub Actions are too high: sometimes you're waiting for 10 minutes just to run into a dumb typo, empty string-evaluated variable or other mishap. There's zero IDE support for almost anything beyond getting the YAML syntax itself right. In regards to writing correct pipelines, I really liked how GitLab CI lets you validate your configuration and even shows how the pipeline would look like, without executing anything, in their web UI: https://docs.gitlab.com/ee/ci/lint.html I think most tools should have something like that, as well as pipeline visualizations - anything to make using them more user friendly! As for the cycle times, if most of what the build or CI action (whatever it might be) needs is already described as "code", you should be able to run the steps locally as well, either with a separate wrapper script for the stuff that you won't get locally (like CI injected environment variables, which you can generate yourself), or with a local runner for the CI solution. This is why I mentioned Drone, which has something nice in this regard: https://docs.drone.io/cli/drone-exec/ But generally, for most simpler pipelines (like the example above), you can even just set up an IDE run profile. In my case, I typically version a few run configurations for JetBrains IDEs, that can build containers for me, run tests and do other things. Sometimes the local experience can be a bit better than what you get on the CI server: if you have any integration tests that automate a browser with Selenium (or a more recent solution), you can essentially sit back and watch the test execute on your machine, instead of having to rely on screenshots/recordings on the server after execution. Of course, much of this would gradually break down, the more complicated your CI pipelines would become. The only thing I can recommend is KISS: https://en.wikipedia.org/wiki/KISS_principle Sadly, this is a non-solution when you're not in control of much of the process. Here's hoping that Dagger and other solutions can incrementally iterate on these aspects and make CI/CD easier in the future! |