[witten]

I took part in evaluating Concourse CI for the needs of my company (30+ devs). While it has amazing CI pipeline capabilities, we ultimately didn't select Concourse because it felt much more like a CI toolbox, requiring some development to put those tools to use. And what we really wanted was more of a turnkey CI product.

Perhaps ironically, we ended up doing some development around the edges of the CI product we ultimately selected (GitLab).

[sytse]

Cool to hear you selected GitLab. We recently added multi project pipeline visualization. That was partially inspired by concourse CI. Is there anything else you would like to see?

[witten]

Interesting you should mention that. Right around the time you implemented multi-project pipeline visualization, we built a bot that listens for GitLab build completion events, hits the (undocumented /unsupported) internal GitLab global code search ElasticSearch index, finds downstream dependencies of the completed build (as declared in setup.py/package.json), and triggers their builds as well.. Taking full advantage of your nifty visualization.

So I think the feature we'd want to make that easier is an actual global code search API!

[grzesiekb]

Thanks for this proposal! It might be a cool feature indeed. I created an issue about it in our issue tracker, see https://gitlab.com/gitlab-org/gitlab-ce/issues/35186.

We do not have immediate plans to ship that yet, but I pinged our product team member there. However we do have plans to ship multi-project pipeline with an "inversion of control".

You will be able to specify pipeline relation with an upstream project, and when someone pushes a commit to it, a downstream pipeline is going to be trigger automatically. See an issue about cross-project dependencies - https://gitlab.com/gitlab-org/gitlab-ee/issues/1681!

[witten]

Awesome, thanks for filing!

[sytse]

Cool! I think we're building this into GitLab itself. I'll ask the CI CD team to comment.

[gaff33]

Please can you allow for any acyclic graph rather than the current stage system? This would allow us to have our long running integration tests run in parallel with shorter tasks!

I wrote an issue for it here: https://gitlab.com/gitlab-org/gitlab-ce/issues/29347

[jacques_chester]

> While it has amazing CI pipeline capabilities, we ultimately didn't select Concourse because it felt much more like a CI toolbox, requiring some development to put those tools to use.

This is actually a brilliant insight. I'd never thought of it that way. And I'm a one-eyed raving Concourse fan.

[mariusmg]

>requiring some development to put those tools to use

The best CI is the one where you write code to build/deploy etc.

Wrangling with some moronic UI exposed by the CI product to abstract code is simply wrong. Don't do CI/CD like this.

[gabrielr]

Right. One of my favorite parts of Concourse is that all the config is declarative, in YAML that teams can (and do) check into a public Git repo (with the secrets kept elsewhere).

That lets teams share Concourse task and resource definitions. Many teams (including mine) publish our pipeline definitions too [0]. This enables "tooling" type teams build components that get a lot of re-use by other teams.

disclaimer: I work for Pivotal.

[0] e.g. code.cloudfoundry.org/cf-networking-ci

[witten]

Some development is fine (and expected). And indeed the goal of "build automation" was what led us to evaluate both Concourse and GitLab. We were previously on Bamboo, which really did require wrangling with a moronic UI.

But I think there's a difference between a mostly complete CI product with API extension points, and a "batteries not included" CI toolbox.