[marcosnils]

Hi there, Dagger contributor here.

One thing that generally is not immediately obvious is all the side benefits that using a programming language actually unlocks.

Need type checking at build time? done.

Need a package manager to version and publish your CI code? done.

Need linting / formatting? done.

Need to import functions from other places? done

Need a test framework? done.

Need a better collaborative IDE? done.

All these things above bring massive improvements to both Dev and Devops engineers that need to interact with build & deploy pipelines daily. Generally it'll be quite difficult for any other configuration or custom language to provide these kind of tooling.

[idoubtit]

I'm sorry, I can't make sense of what you wrote, those "side benefits that using a programming language [in CI] actually unlocks". Could you please give a concrete example?

> Need type checking at build time? done.

It's not unlocked by Dagger since every CI can already do this. And it's almost always about launching a (shell) command. How is that line easier in Go/any-language than in yaml?

> Need to import functions from other places? done

You mean "import functions into the CI code"? Why would I need that? It's usually possible through plugin systems, but I've never felt the need to put complex code into CI actions.

> Need a test framework? done.

Is that a test framework to test the CI code from within the CI? I hope I'll never want one! I guess I completely misunderstood...

On the other side of those "benefits", I can smell problems. For instance, when Docker images and Compose systems are built from the Dagger API. If the need for some advanced features comes in, this extra layer could prove costly.

[jahsome]

I believe they're describing the development workflow for CI/D specifically.

It sounds like you're not in need for most of the use cases but each of these points presents decent value at certain scales.

[shepherdjerred]

The parent comment was saying that those benefits could be applied to your build/CI scripts.

Consider a complex (1000+ line) YAML GitHub action that involves building, testing, linting, deploying, releasing, etc.

They are pretty common and are extremely hard to both read and write.

Dagger fixes those issues for complex pipelines since you get to use a "real" language.

> You mean "import functions into the CI code"? Why would I need that? It's usually possible through plugin systems, but I've never felt the need to put complex code into CI actions.

This might just not be a use case for you, which is okay. This is just a general property of modern programming languages, and the result of Dagger being plain Go/TypeScript/etc.

> Is that a test framework to test the CI code from within the CI? I hope I'll never want one!

At AWS we could only deploy on certain days, e.g. we wanted to ensure that our pipelines didn't deploy to too many regions at once, that we didn't deploy during holidays, or during peak shopping times of the year.

We had unit tests for our pipelines that ensured that the pipeline had the proper rules to enforce the above. I would imagine there are many other use cases for testing your pipeline.

[melezhik]

> Consider a complex (1000+ line) YAML GitHub action that involves building, testing, linting, deploying, releasing, etc.

If one has a 1000+ yaml pipeline - this is not because they did not hear about dagger, but because they haven’t done proper decomposition and refactoring where all the complexity is abstracted away from pipelines dsl into proper general purposes language libraries

[shepherdjerred]

I completely agree, which is why Dagger is perfect for medium/large projects.

I think Earthly is better for small projects with a simple build system since it's so much simpler to write and understand. I think Dagger is a worthwhile investment once your Earthfile is >200-400 lines.

[marcosnils]

> I completely agree, which is why Dagger is perfect for medium/large projects.

Initially that was the case, but with the upcoming work on modules (aka project Zenith), hopefully this changes. After all, IMO the ultimate aspects that will make it super easy to adopt is the ecosystem and learning how to use the tool. The later is mostly tackled by the "pick your favorite language" story; the former is where we hope modules can help users the most.

[melezhik]

You can have all that by using yaml based pipeline framework _together_ with building blocks written on general purpose languages

[shykes]

That’s how Dagger works. You use Dagger to make those building blocks cleaner, faster, and more portable. Then you can run them directly, or from your favorite CI configuration.