[mordocai]

I'm not a gitlab employee or anything, but I'm curious: How is gitlab CI not flexible enough for you?

I mean, it basically just runs arbitrary docker containers. From my mostly amateur (when it comes to this, i'm a professional developer but I don't manage our CI system directly) point of view it seems like the gitlab CI is more flexible than jenkins not less.

[mikepurvis]

The strong coupling between repos and CI jobs is a huge obstacle— for a multi-repo codebase, you need to have triggers that are outside of "a new commit has been pushed".

The reporting is a big problem too— Jenkins has plugins which format and report on JUnit-style reports, including displaying images or other assets generated by tests.

Configuration. The config needing to be committed to the repo is an issue. For many use cases including ours, it's way better to have the CI configuration stored centrally somewhere, and deployed using Jenkins Job Builder (or the various other similar tools which exist for Jenkins).

Executor control. Jenkins lets you do things like put N slots on a buildslave and then have a job occupy M of them, as well as controlling what jobs can run where, and what the priority of different jobs is. This is critical for ensuring overall sanity and efficiency in the pipeline.

Jenkins isn't perfect, but it happens to be a better fit for our needs, and these four factors are all a big part of that.

[sytse]

For multiple projects you can use build triggers https://docs.gitlab.com/ce/ci/triggers/README.html

We're considering parsing JUnit style tests https://gitlab.com/gitlab-org/gitlab-ce/issues/22098

We think storing the configuration in the repository makes it easier to collaborate on it (new branch is tested in the new way, master in the old way). Can you share your use case for this?

You can specify how many jobs you want GitLab Runner to run. It can even spin up more VMs automatically if you want that. You can bind jobs to runners with labels and specific runners https://docs.gitlab.com/ee/ci/runners/README.html

[mikepurvis]

Our use-case is perhaps a bit unique, but ROS has a federated development model, meaning literally hundreds of repos. See the following file which maps repos to package names and versions:

https://github.com/ros/rosdistro/blob/master/indigo/distribu...

These get generated into thousands of jobs: http://build.ros.org/view/All/

It would be totally unrealistic to rev job configurations by making commits into the individual repos (and in the public open source case, actually impossible— the maintainers of that community build infrastructure don't have write access to many of the repos— it's equivalent to a Debian distro using DSC files to grab tarballs).

[sytse]

Thanks for the use case and the links. I must admit I never saw a 14234 line yaml configuration file before.

The jobs page is not loading for me right now.

I think that in general developers and the build engineers should both have commit access. But I see your dilemma and think we need to have a conversation to see what the possibilities are.

[Snappy]

Great feedback, thanks!

This isn't exactly what you asked for, but we're seriously considering letting you "include" CI configuration from a central location.

https://gitlab.com/gitlab-org/gitlab-ce/issues/20868

[elygre]

As one example, GitLab CI does not have a good story with handling unit test failures from e.g. gradle.

You can choose to keep the build output from a failed build, which you can download as a zip file, unpack locally, and look at the reports. This user experience makes test reports mostly useless.

At the same time, they have test coverage badges, where test output is parsed to provide coverage. To me, that's really weird: You don't get proper access to test failures, but lots of work into test coverage.

(I have no knowledge of this, but I suspect that being a Ruby-shop, they have little experience with working with java projects, and therefore don't really understand how the toolchain works. This is all guesswork, though).

In the end, we cannot give up Jenkins. We need to know the result of our tests, and see the test output!

[sytse]

I totally agree that we can do better showing test output and we're working on that in https://gitlab.com/gitlab-org/gitlab-ce/issues/22098

[sytse]

As you indicated we hope that GitLab CI covers all use cases but we're always interested in learning things we can improve.

[elygre]

I added a comment to the parent, regarding test reporting. It's a showstopper for us.

[mitchty]

Just running docker containers is basically a show stopper for certain use cases. Not every use case can even fit into a docker container. E.g. testing kernel modules.

[sytse]

Totally agree that not everything works on Docker. GitLab Runner works great on metal (Linux, OSX, Windows, RPi, etc.), no need to use Docker.

[merb]

most people probably miss JUnit-style reports.

[sytse]

Yep, see https://gitlab.com/gitlab-org/gitlab-ce/issues/22098 for the feature proposal.