[chillfox]

Absolutely, I feel like most CI systems are much harder to configure than necessary.

If I were to build my own CI system, then it would have these features.

- Simple config format, ideally flat and not nested, so maybe toml.

- Ability to run any step, task, job, pipeline locally. It should be possible to do something like `ci run -t build` and have it execute the build task locally. This would greatly help with building CI pipelines and make it possible to just use the CI tool as the local build tool as well instead of having to essentially setup the same thing twice.

- Easy to self host, so not a million micro-services, but just one (or max 3) executable.

- Modular, it should be possible to extend it with plugins using webassembly or containers. Look at how Concourse CI does this for ideas.

Edit: I would pay for a license to something like that, maybe not a lot but I could justify up to $200 as a once off for a self-hosted solution like that if it's well made.

[xwowsersx]

This doesn't have everything you mention, but I recently learned about https://dagger.io/ which allows you to write your pipelines as code (Go, Python, etc) and then run them anywhere in containers. I always hated that in order to test my pipelines, I actually have to trigger it on some special machine (the CI server).

[chillfox]

Dagger looks interesting, I will check it out.

[dosisod]

I think these are all great points! I have came across Concourse CI, but have not looked too deeply into it yet.

I hadn't thought to use TOML for configuring CI pipelines, I'm curious what that might look like. TOML is indeed very flat, so it would be interesting to see what the equivalent TOML pipeline looks like compared to a YAML one.

[chillfox]

I run Concourse CI for my personal stuff with the code in Fossil. It works decently enough, I like that it's somewhat extensible, but I have hit use cases that it can't really do like building an Alpine Linux cloud image.

As for YAML vs TOML, I am mostly sick of YAML, and I really like using TOML to configure Caddy, so I thought it might be good enough for CI config. I feel like it would be worth exploring, but maybe something slightly more complex or custom would be required.

[mike_hearn]

Sounds a bit like TeamCity, except for the local execution. It has:

- Either GUI based configuration (serialized to XML), or Kotlin. So you can express configuration with a real programming language tracked with version control, or just use the GUI to define it.

- Can self host. There's one program to install for the master, and one for the agents.

- Can be extended with plugins. Not webassembly based, but those are theoretical anyway currently due to lack of better-than-C interfaces. You can write JVM plugins instead and get full GCd OOP.

The main thing it's lacking is the ability to run things locally. But then again, usually a TC step is just invoking a build system or script anyway so it's not that big of a deal.

[P5fRxh5kUvp2th]

In addition, it would need to try and solve for the 80% witch hatches for those that need the last 20%.

Part of the problem with current CI/CD solutions is trying to solve for everything.

And I think the local thing would be a killer if for no other reason than the ability to debug it locally.

[chillfox]

That's kinda my idea with plugins, it should be possible to develop a plugin doing your custom thing if you need something in that last 20%.

Concourse CI has a design documented for a better resource/plugin system than what they currently have implemented that's worth checking out. But as far as I am aware they are really far from actually implementing it.