George V. Reilly

Jenkins #6: Miscellenea

[Pre­vi­ous­ly published at the now defunct MetaBrite Dev Blog.]

A collection of mis­cel­la­neous tips on using Pipelines in Jenkins 2.0.

#6 in a series on Jenkins Pipelines

En­vi­ron­ment Variables

Use the withEnv step to set en­vi­ron­ment variables. Don’t manipulate the env global variable.

The confusing example that you see in the documents, PATH+WHATEVER=/something, simply means to prepend /something to $PATH. The +WHATEVER has no other effect.


The withEnv step should not be used to introduce secrets into the build en­vi­ron­ment. Use the with­Cre­den­tials plugin instead.

    [$class: 'StringBinding', credentialsId: 'GPG_SECRET', variable: 'GPG_SECRET'],
    [$class: 'AmazonWebServicesCredentialsBinding',
     credentialsId: '0defaced-cafe-f00d-badd-0000000ff1ce',
     accessKeyVariable: 'AWS_ACCESS_KEY_ID',
     secretKeyVariable: 'AWS_SECRET_ACCESS_KEY']
]) {

Tip: Use the Pipeline Syntax Snippet Generator from your Jenkins project to figure out the exact invocation for your cre­den­tials.

If you need to display en­vi­ron­ment variables for debugging purposes, you can use something like this Python script instead of sh 'env | sort', which can leave secrets in cleartext in your logs. Note that with­Cre­den­tials will mask values that it knows about.

#!/usr/bin/env python

import os, re

BLOCKLIST_RE = re.compile(r'^(.*_)?(PASSWORD|AUTH|TOKEN|SECRET|KEY)(_.*)?$')

for k in sorted(os.environ.keys()):
    v = os.environ[k]
    if BLOCKLIST_RE.match(k) and len(v) > 2:
        v = '*{}{}{}*'.format(v[0], len(v)-2, v[-1])
    print('{}={}'.format(k, v))

Cleaning Up Workspaces

You may need to clean up your workspace. Don’t remove the .git directory.

sh "ls -A1 | grep -v "^\\.git\$" | xargs rm -rf"

Editing Jenk­ins­file

Be sure to put #!groovy as the first line in your Jenk­ins­file, as a hint to your editor to use Groovy syntax high­light­ing.

We use PyCharm a lot, since we’re a Python shop. However, despite being written in Java, PyCharm’s support for Groovy is mediocre. You’re expected to buy JetBrains’ IntelliJ IDE.

Vim has built-in support for Groovy. Atom with the language-groovy package works great. Many other editors also support Groovy.

Stashes and Artifacts

Use stash and unstash to get artifacts across nodes within a pipeline. If you’re using the parallel step, some of your branches will start out with an empty workspace, and unstash is the easiest way to prime the workspace.

If you need to transfer artifacts between different builds, you’ll need to use archiveArti­facts and Cop­y­Arti­fact.

In project build-1:

archiveArtifacts artifacts: '', fingerprint: true

In project build-2:

step ([$class: 'CopyArtifact',
       projectName: 'build-1',
       filter: '',
       selector: [$class: 'StatusBuildSelector']

Note that the Sta­tus­Build­S­e­lec­tor selector is picking the artifact from the last successful build of build-1.

You can find the various selectors in the Cop­y­Arti­fact source. If you dig into the tests, you can probably figure out how to use the other selectors.

In build-1, you can use build job: 'build-2', wait: false to kick off build-2.


Here are some tricks for debugging.

Local Groovy

Run Groovy locally to debug syntax issues and Groovy functions.

timestamps and ansiColor

Use the timestamps plugin to get timestamps to appear in the Jenkins log. This is so useful that it should be the default.

The ansiColor plugin will generate transform colored console output into colored log output.

node('ubuntu') {
    timestamps {
        ansiColor('xterm') {
            stage("Source Checkout") {
                checkout scm

Job Debugging

Instead of trou­bleshoot­ing build problems in a slow heavy­weight job, create a new light­weight job that isolates the problem. You may get your iteration time down to a minute or so, instead of many minutes.

Tem­porar­i­ly use "Pipeline script" instead of "Pipeline script from SCM", so that you can try out changes more quickly. Note that checkout scm won’t work unless the Pipeline script comes from SCM.

The Replay command in the left menu lets you edit the Pipeline script and rerun it.

sh Errors

You can use cat -n $0 to echo the in­ter­po­lat­ed sh script to the log.

blog comments powered by Disqus
Review: I Shall Wear Midnight » « Review: Kill Me Three Times