[Dries007]

Shameless self-plug:

I very much live by the "fix all warnings before you commit" (or at least before you merge), so I have Shellcheck and a bunch of other linters set up in my pre-commit configurations. But the majority of the shell in most of my projects ends up embedded in .gitlab-ci.yml files, where it's hard to check. So I made a wrapper that does that automatically: https://pypi.org/project/glscpc/.

It uses the Shellcheck project and some magic to give you the Shellcheck remarks with (mostly) accurate line-numbers.

[brirec]

I’d love to see a project that would do this, but more generally.

I don’t use GitLab CI, but I do use a good handful of other file types that essentially inline shell scripts. Dockerfiles, GitHub Actions, and Justfiles, just to name a couple.

Usually, and almost exclusively for the sake of ShellCheck, I make a point of putting anything more complex than a couple of commands into their own shell scripts that I call from the inlined script in my Dockerfile.

(This pattern also helps me keep my CI from being too locked into GitHub Actions)

[vinnymac]

I’d rather see CI services add a mode that would enforce all scripts must live in separate files, rather than inline.

It’s really not necessary to support inline except for single lines that are very short (under 30 chars).

[plorkyeran]

I am not generally a fan of Xcode Cloud's design, but this is one thing I think it gets very right. Rather than let you specify the build actions in the CI settings, it invokes scripts with fixed names in the `ci_scripts` directory of your repo if they exist. There just isn't a mechanism for creating jobs which aren't something that you can build locally and test independently of Xcode Cloud.

[Dries007]

I would say: stick to straight command sequences with variables, avoid if/for/while/functions etc. Subshells and pipes only for trivial things.

Setting something like a 30 char limit will inevitably lead to code-golfing to fit a mess into the 30 char limit.

But that becomes harder to automatically check, that's why you should still have good peer reviews based on written standards somewhere.

[Piraty]

so much this. it will help CI not become a holy, super-hard-to-debug, unreproducible mammoth. write scripts, call them from CI

[marcosnils]

Hi there, Dagger contributor here. We're solving this exact same problem by allowing you to encode your CI/CD pipelines in your favorite programming language and run them the same way locally and/or any CI provider (Gitlab, Github Actions, Jenkins, etc).

We're very active in our Discord server if you have any questions! https://discord.gg/invite/dagger-io

[Dries007]

We've been experimenting with Dagger here, as part of the alternative to writing glscpc actually, but I'm not convinced it's ready to replace Gitlab CI.

Dagger has one very big downside IMO: It does not have native integration with Gitlab, so you end up having to use Docker-in-Docker and just running dagger as a job in your pipeline.

This causes a number of issues:

It clumps all your previously separated steps into a single step in the Gitlab pipeline. That doesn't matter too much for console output (although it does when your different steps should run on different runners), but is very annoying if you use Gitlab CI's built in parsing of junit/coverage/... files, since you now have extra layers of context to dig trough when tests fail etc. Plus not all of these allow for multiple input files, so now you have to add extra merging steps.

If your job already uses Docker-in-Docker for something, you have to be careful not to end up with Docker-in-Docker-in-Docker situations, or container name conflicts if you just pass through the DOCKER_HOST variable.

The one thing that would make this worth it is being able to run the pipelines locally to debug, but I've just written quick-and-dirty scripts to do that every time I've needed it. For example, running the test job in our pipeline on every Python version: https://gitlab.com/Qteal/oss/gitlabci-shellcheck-precommit/-...

[marcosnils]

> but I'm not convinced it's ready to replace Gitlab CI.

The purpose of Dagger it's not to replace your entire CI (Gitlab in your case). As you can see from our website (https://dagger.io/engine), it works and integrates with all the current CI providers. Where Dagger really shines is to help you and your teams move all the artisanal scripts encoded in YAML into actual code and run them in containers through a fluent SDK which can be written in your language of choice. This unlocks a lot of benefits which are detailed in our docs (https://docs.dagger.io/).

> Dagger has one very big downside IMO: It does not have native integration with Gitlab, so you end up having to use Docker-in-Docker and just running dagger as a job in your pipeline.

Dagger doesn't depend on Docker. We're just conveniently using Docker (and other container runtimes) as it's generally available pretty much everywhere by default as a way to bootstrap the Dagger Engine. You can read more about the Dagger architecture here: https://github.com/dagger/dagger/blob/main/core/docs/d7yxc-o...

As you can see from our docs (https://docs.dagger.io/759201/gitlab-google-cloud/#step-5-cr...), we're leveraging the *default* Gitlab CI `docker` service to bootstrap the engine. There's no `docker-in-docker` happening there.

> It clumps all your previously separated steps into a single step in the Gitlab pipeline.

It's not generally how we recommend to start, we should definitely improve our docs to reflect that. You can organize your dagger pipelines in multiple functions and call them in separate Gitlab jobs as you're currently doing. For example, you can do the following:

  ```.gitlab-ci.yml
  build:    
    script:
      # no funky untestable shellscripts here, but calls to \*real\* code
      - dagger run go run ci/main.go build
  test:
    script:
      # no funky untestable shellscripts here, but calls to \*real\* code
      - dagger run go run ci/main.go test
  ```

This way, if your pipeline currently has a `build` and `test` job, you still keep using the same structure.

> but is very annoying if you use Gitlab CI's built in parsing of junit/coverage/... files, since you now have extra layers of context to dig trough when tests fail etc

You can also still keep using these. The only thing you need to be aware is to export the required test / coverage output files from your Dagger pipelines so Gitlab can use them to do what it needs.

> but I've just written quick-and-dirty scripts to do that every time I've needed it.

This is what we're trying to improve. Those quick-and-dirty generally start very simple but they become brittle and very difficult to test by other engineers. Yes, you could use docker or any container-like thing to enable portability, but you'll probably have to write more scripts to glue all that together.

Quoting one of our founders:

"Our mission is to help your teams to keep the CI configuration as light and "dumb" as possible, by moving as much logic as possible into portable scripts. This minimizes "push and pray", where any pipeline change requires committing, pushing, and waiting for the proprietary CI black box to give you a green or red light. Ideally those scripts use containers for maximum reproduceability. Our goal at Dagger is to help democratize this way of creating pipelines, and making it a standard, so that an actual software ecosystem can appear, where devops engineers can actually reuse each other's code, the same way application developers can."

[iamjackg]

Absolutely agree. The main downside of that pattern is that it doesn't work with jobs included from other projects in GitLab CI, since the job runs in the context of the project that imported it and therefore can't find the script in its original repo. Huge bummer.

[vinnymac]

Bundle up configuration and scripts, and now we have containerized CI infrastructure.

[Dries007]

Generalizing this is non-trivial (I tried initially) but I'm sure others can build in the same principles.

I think this comes close for Dockerfiles: https://hadolint.github.io/hadolint/ Just have to write a pre-commit hook for it.

[verdverm]

You might like Dagger, building images with code, uses the same buildkit engine under the hood

No more linear Dockerfile, use the powers of your preferred language

[stouset]

This is the way

[l0b0]

High five - I did something similar very recently[1]; usage[2]:

  - repo: https://gitlab.com/engmark/shellcheck-gitlab-ci-scripts-hook
    rev: 174fda8f384db229aca5b40380e713d0c25a1cb9 # frozen: v1
    hooks:
      - id: shellcheck-gitlab-ci-scripts
        files: ^\.gitlab-ci\.yml$

[1] https://gitlab.com/engmark/shellcheck-gitlab-ci-scripts-hook

[2] https://gitlab.com/engmark/root/-/blob/9f7d9b93c2297d0b170e5...

[iamjackg]

Oh, that's really cool. I was trying to solve this from a different perspective a while ago: I wanted to add some pre-processing that would take a "normal" shell script and render it to the `script` part of the corresponding job at build time, the advantage being that you still have everything self-contained in the gitlab-ci job.

I stopped working on it because dealing with shell shenanigans in the GitLab CI runner environment is such a miserable experience that we're in the process of moving all our jobs to python scripts.

[Dries007]

Yea, Pythonifying the scripts is also generally my preferences the moment they become somewhat complex. But even then it's nice that you can be reasonably sure you're not forgetting quotes around variables or using bash constructs where you only have sh.