[jillesvangurp]

I use matrix tests with github actions to test my kt-search client with different versions of elastisearch and opensearch. Pretty easy to set up: https://github.com/jillesvangurp/kt-search/blob/master/.gith...

Basically it fires up elasticsearch using docker-compose and then the integration tests run against that. You could use a similar strategy to test different feature flag combinations.

For some of our private projects, we use kts to generate the github action yaml files using this: https://github.com/krzema12/github-workflows-kt

Well worth checking out if you have more complex workflows. Yaml is just horrible in terms of copy paste reuse. Also nice to get some compile time safety and auto complete with our action files.

[diarrhea]

Pretty sure the parent meant testing the pipelines themselves, end to end.

The tooling around pipelines is awful. A single typo in some variable’s name in a later stage can take minutes to catch. The feedback cycles are very long (cloud machines are much slower than local ones) and IDE tooling is bare-bones.

Just give me one large Python file with some library to manage common actions (building up the job DAG, accessing pull requests, easy shell access, …). We’d have refactoring, Turing completeness, type safety and so much more. A core downside would be managing the complexity of DevOps scripting going berserk. Personally I’d prefer that trade off.

[Denvercoder9]

> The feedback cycles are very long (cloud machines are much slower than local ones) and IDE tooling is bare-bones.

I'm continuously amazed that none of the major CI providers offer standalone tooling to run and debug your CI pipelines locally. Seems like it'd be a killer feature for anyone working with complex pipelines.

[verdverm]

The local & CI parity is one of the main points for https://dagger.io

[theSage]

I've been building Jaypore CI for exactly this tradeoff. The config is a single python file and I run jobs using a git hook on my own laptop.

I'd love some feedback on what else I could add to this project to make life easier for people.

https://www.jayporeci.in/

[jillesvangurp]

Testing the pipelines is a thing as well. But pretty tricky to set up. You basically need to run dockerized actions locally for that. Not impossible and I've seen some attempts to do that. But I can't really justify spending a lot of time on this stuff.

The kotlin scripting support for github actions that I mentioned addresses things like typos, refactoring, and IDE tooling. Try it, it's pretty nice and easy to use. We actually have an integrity check as part of our build that runs the kts script to verify the yaml file stored in the repository is consistent with what the script generates.

[raptorraver]

Exactly this. We are having quite complex pipelines and almost weekly basis we introduce some regression bugs there or change the rules so that they don't work anymore properly. It takes around 20 minutes to run the whole pipeline and of course some jobs are only triggered when merging to master etc. The development flow is painful and slow (although we have found some tooling to run some parts of our pipelines locally).