My first approach was as an individual, which was quite difficult, because it took me a while to work out the differences between jobs and tasks and how to lay it all out. Lots of copying and pasting from other pipelines I studied.
At this point pretty much every team in Pivotal's Cloud Foundry division is running a Concourse pipeline, including the one I belong to. What's been interesting is how each team is experimenting with patterns that Concourse makes possible.
For example, I and my peers are now turning various things into "executable documents" -- the terminology sucks at the moment. Think of all the stuff buried in READMEs and wikis and long-forgotten cron jobs. How to build that special docker image that you only update every few months. The database backup. Keeping the blue-green deploy codepath warm.
When we find another one of these, we now encode it into our pipeline. That way, if I need to find it, it's there. And if it goes bad, the pipeline definition points me at where to go looking for everything of interest.
Another pattern that is emerging for us is "enforce project invariants". For example, we have multiple repos, so multiple Gemfiles and Dockerfiles, all of them with ruby versions set. At the front of our pipeline there is a little gateway to check that these are all identical. If not, it prints a table of versions found, so again, I know instantly where to go look. We have various other little invariants that turn days of insane debugging into a few seconds of sanity-checking.
None of this is novel. What Concourse lets me do is hoist good design practices out of code and apply them to CI. The final feedback loop I rely on can itself now be checked in, broken apart according to SOLID principles and even (my colleagues in Buildpacks are pioneering this) unit tested. Some colleagues in London have designed a fully generic pipeline that can be configured at runtime to build any of the five products they manage.
My first approach was as an individual, which was quite difficult, because it took me a while to work out the differences between jobs and tasks and how to lay it all out. Lots of copying and pasting from other pipelines I studied.
At this point pretty much every team in Pivotal's Cloud Foundry division is running a Concourse pipeline, including the one I belong to. What's been interesting is how each team is experimenting with patterns that Concourse makes possible.
For example, I and my peers are now turning various things into "executable documents" -- the terminology sucks at the moment. Think of all the stuff buried in READMEs and wikis and long-forgotten cron jobs. How to build that special docker image that you only update every few months. The database backup. Keeping the blue-green deploy codepath warm.
When we find another one of these, we now encode it into our pipeline. That way, if I need to find it, it's there. And if it goes bad, the pipeline definition points me at where to go looking for everything of interest.
Another pattern that is emerging for us is "enforce project invariants". For example, we have multiple repos, so multiple Gemfiles and Dockerfiles, all of them with ruby versions set. At the front of our pipeline there is a little gateway to check that these are all identical. If not, it prints a table of versions found, so again, I know instantly where to go look. We have various other little invariants that turn days of insane debugging into a few seconds of sanity-checking.
None of this is novel. What Concourse lets me do is hoist good design practices out of code and apply them to CI. The final feedback loop I rely on can itself now be checked in, broken apart according to SOLID principles and even (my colleagues in Buildpacks are pioneering this) unit tested. Some colleagues in London have designed a fully generic pipeline that can be configured at runtime to build any of the five products they manage.
tl;dr fuck yeah Concourse.