George V. Reilly

Jenkins #1: Migrating to Pipelines

[Pre­vi­ous­ly published at the now defunct MetaBrite Dev Blog.]

The MetaBrite dev team migrated most of their builds from Atlassian’s Bamboo Cloud to Jenkins Pipelines in late 2016/early 2017. This is a series of blog posts about that experience.

Jenkins Pipeline Series

The series so far:


For three years, we used Atlassian’s hosted Bamboo Cloud service to build and deploy most of our code. In the summer of 2016, Atlassian announced that they were going to dis­con­tin­ue Bamboo Cloud on January 31st, 2017.

We looked around for a suitable re­place­ment. We did not find anything would work well for us. We had re­quire­ments that were—­sur­pris­ing­ly—hard to satisfy. Much of our code is in one large GitHub repository, from which we build several very different ap­pli­ca­tions. Many of the CI systems that we rejected were designed to build only one project per repository. Another hard re­quire­ment was that a push to the master branch in this repo should not trigger a build of every ap­pli­ca­tion; only the ap­pli­ca­tions in the affected di­rec­to­ries should be built. We also didn’t want to host our own build system, preferring to pay for a service that considered hosting CI to be one of their core com­pe­ten­cies. We liked some of the systems that we evaluated, but none of them satisfied our needs.

Failing to find a new system that we liked, we re­luc­tant­ly ex­per­i­ment­ed with hosting Bamboo Server ourselves, reasoning that it should be fairly easy to get our Bamboo Cloud builds working on Bamboo Server. It wasn’t. Getting Bamboo Server even half working was painful and un­sat­is­fac­to­ry. Time for Plan C.

We took a closer look at Jenkins; more specif­i­cal­ly, at CloudBees‘ hosted offering. Some of us had previously used Jenkins Free-Style Builds at other jobs in the past and were un­en­thu­si­as­tic about doing so again. In fact, we had already been customers of CloudBees for a couple of years, as most of our Android builds were built there.

We initially ex­per­i­ment­ed with Free-Style builds and found them no more pleasing than we remembered. Hesitantly, we tried the Pipeline builds (formerly the Workflow Plugin) that formed the heart of last year’s Jenkins 2.0 release.

With Jenkins Pipelines, we finally found something that made us happy. But there was a long, steep learning curve and we haven’t reached the summit yet. This series of posts describes much of our hard-won knowledge. Had we found similar posts written by someone else before we started, we would have saved ourselves con­sid­er­able time.


The Pipeline is the heart of Jenkins 2.0. The tra­di­tion­al Free-Style build has been supplanted by a more powerful mechanism that supports a rich Domain-Specific Language (DSL) and a variety of other features.

Having our build system be under source control was almost enough to sell us on Pipeline. After Bamboo, where a build had to be configured across multiple screens, tabs, and forms, having everything in a single Groovy file was appealing. (A few settings do still need to be configured in a Jenkins webform.) Being able to use an expressive language like Groovy as the basis of the DSL leads to compact, terse, main­tain­able, and reusable func­tion­al­i­ty.

This series of posts is not intended to be a com­pre­hen­sive in­tro­duc­tion to Pipelines. These resources may help:

The Pipeline doc­u­men­ta­tion is, bluntly, not very good. It’s frequently confusing and often incomplete. You need to search and to be willing to paw through Java code in GitHub repos­i­to­ries.


The work on which this series is based was jointly done with Adam Porad. Several of CloudBees’ support engineers were very helpful, including Owen Wood, Allan Bur­da­jew­icz, Travis Sweet, and Adrien Lechar­p­en­tier.

blog comments powered by Disqus
Review: Gone, Baby, Gone » « Jenkins #2: EC2 Slaves