[0x1F8B]

What I would do is find a way to decouple your test running system from the lifecycle tooling. Develop a custom solution that run CI pipelines in k8s and collates results and passes them upstream to the lifecycle tooling whether it's GitHub Actions or Jenkins or whatever. By doing this you free your pipelines from lock-in and can utilize cheap compute (DigitalOcean, GKE pre-emptibles, etc).

Take a look at Tekton CI/CD for a good primitive that you can build on.

[MaulingMonkey]

Just having your CI steps run scripts that install/build/deploy/??? already goes a long way to decoupling from your CI solution, and can be a lot faster than spinning up a custom image for your CI build. Custom images don't necessairly do much for fixing the last 10% either - I'll typically want CI-specific:

1. Build matricies. I was running windows builds on appveyor and linux builds on travis for a long time (faster builds / iteration / feedback.) Regression testing against multiple rustc versions can be done from within a script, but doing so via CI-configuration makes for easier to read CI results that flag individual builds as failed instead of an entire script that you have to go log diving into.

2. Custom runners. Be it custom hardware or licensing-burdened custom software, I frequently need to own the actual CI hardware, and using the native solutions for CI runners is typically easier than merely configuring whatever custom infrastructure I might come up with.

3. Images. Using prewarmed cloud-ci-specific images is often faster to spin up.

[toomanybeersies]

Buildkite is BYO infrastructure. It's worked really well for us, as we have a fairly idiosyncratic build process/test suite that out of the box solutions like GH Actions or CircleCI don't really cater to.