[maratc]

I really scratch my head when I read your comment, as nothing of this is a real issue in my Jenkins.

> bunch of random Jenkins servers

Either PXE boot from an image, or k8s from an image, have a machine or pod rebooted/destroyed after one job. Update your image once a month, or have a Jenkins job to do that for you.

> Authentication, authorization, access control

Either use LDAP or Login via Github, and Matrix security plugin. Put all "Devops" group into admins, the rest into users, never touch it again.

> configuration

CASC plugin and seed for jobs, and/or Helm for just about everything else.

> env vars and secrets

Pull everything from Vault with Vault plugin.

> as a sysadmin Jenkins will suck the life out of you

I spend about 1-2 hours a week managing Jenkins itself, and the rest of the week watching the jobs or developing new ones.

[0xbadcafebee]

Well one issue is, CasC isn't enough. You often have to write JobDSL to get around some limitation in CasC, and sometimes Groovy for limitations in the other two. If you want to manage access control (and you choose the correct Auth plugin, and figure out how to configure it), often you need an admin to make changes in both the Jenkins server and your backend AuthNZ system. Then there's the "seed job vs not-seed-job" weirdness that doesn't exist with GHA. And building the (hopefully containerized) Jenkins server, Jenkins build agents, etc will depend on your infrastructure provider, but still usually requires you to get your hands dirty. There are many, many more layers to the onion with Jenkins, and it's just not worth all that overhead for what should be "git clone && build && deploy" - which GHA does much simpler, right where your code lives, without you needing to maintain anything.

And this is if you get to manage it! Often there's 5 different random Jenkins servers set up by different teams, all of which are EOL and rife with security holes, and they expect you to fix them when they break, nobody version controls their configs or backs them up (they haven't even heard of CasC and have no interest in using it), and your boss says you can't say no, and also you can't upgrade them/take them over. I've seen million-dollar products which are completely dependent on over a thousand Jenkins jobs on an out-of-date Jenkins server, so complex and intertwined it couldn't be replaced.

If it were up to me, I would replace most CI with Drone.io (or Woodpecker CI if it ever gets feature parity). Now that's a dead simple CI system.

[maratc]

My issue with GHA and other "dead simple" systems is that my CI is complicated. Having a real programming language for stuff like "calculate what date it was a week ago" or "concatenate these three strings but only under some conditions" or "parse the output and build an object out of it" is really helpful while a bastardised YAML-based Jinja template simply can't hold up.

But yeah, if all there is to do is "git clone && build && deploy" then Jenkins is an overkill and it probably wasn't warranted in the first place.

[0xbadcafebee]

For complex logic I don't rely on the CI system; I've been burned too many times. I shell out to an external program and have it return an output variable, and I just do "if $foo = y then blah" in the CI's DSL (and I keep those tests to a minimum; rather have more separate jobs than one complex job). Often I will put everything in a dedicated build tool (Make or similar) so I can run it from my laptop or CI, and any change to logic only happens in one place. It's adding an abstraction, but the end result is I write the CI job once and never touch it again. For flexibility I add parameters to the CI job.

[maratc]

> For complex logic I don't rely on the CI system

I do that too, until my complex logic belongs to the CI system and nowhere else.

As an example:

    It's 2 am now and Jenkins needs to run some (but not all) nightly tests[0]. To figure out which, let's bring the source code and analyse the configuration file, disregarding anything that doesn't need to run at this hour. Once we have the plan for which tests to run, let's figure out what we need to build[1]. Also, let's see what is the status of the pool running the tests[2] so we can decide on a tests parallelisation strategy for this run. When we have a plan, let's build and test, keeping an eye on the triggered tests. When all these have finished, analyse the failures[3], create tickets for the failures[4] and prepare a report to be sent.

I wouldn't be able to express all this in YAML.

[0] other "nightly tests" run at 1, 3, 4 etc.

[1] this is mapped in the configuration file too.

[2] this is internal to Jenkins

[3] same

[4] this involves finding the "responsible person," so a lot of API calls

[0xbadcafebee]

Sounds like you're using cron as a complex job queue! A lot of teams get there eventually, and either 1) keep hacking on cron/jenkins/etc to make this work, 2) invent their own queueing tool (NIH syndrome; been done many times before, there is nothing new to make here), or 3) use a purpose-built solution for this. Airflow is the old-and-busted solution; the new hotness is newer generations of the same concept (Prefect/Dagster, Luigi, Temporal). But often sticking to your existing thing is cheaper; depends how much custom engineering you want to invest.

Fwiw, I do believe you can do this in GHA, but you may need to call their API from your workflow. In addition, their replacement for Groovy is to run an action which lets you embed Javascript/Typescript and call their SDK. It sucks, but so does Groovy! ;-)