Declarative Pipeline for Maven Projects

    This is a guest post by Liam Newman, Technical Evangelist at CloudBees.

    Declare Your Pipelines! Declarative Pipeline 1.0 is here! This is first in a series of blog posts that will show some of the cool features of Declarative Pipeline. For several of these posts, I’ll be revisiting some of my previous posts on using various plugins with (Scripted) Pipeline, and seeing how those are implemented in Declarative Pipeline.

    To start though, let’s get familiar with the basic structure of a Declarative Pipeline by creating a simple Pipeline for a Maven-based Java project - the Jenkins JUnit plugin. We’ll create a minimal Declarative Pipeline, add the settings needed to install Maven and the JDK, and finally we’ll actually run Maven to build the plugin.

    Set up

    With Declarative, it is still possible to run Pipelines edited directly in the Jenkins web UI, but one of the key features of "Pipeline as Code" is checking-in and being able to track changes. For this post, I’m going to use the blog/add-declarative-pipeline branch of my fork of the JUnit plugin. I’m going to set up a Multi-branch Pipeline and point it at my repository.

    JUnit Multi-branch Pipeline Configuration

    I’ve also set this Pipeline’s Git configuration to automatically "clean after checkout" and to only keep the ten most recent runs.

    Writing a Minimal Pipeline

    As has been said before, Declarative Pipeline provides a more structured, "opinionated" way to create Pipelines. I’m going to start by creating a minimal Declarative Pipeline and adding it to my branch. Below is a minimal Pipeline (with annotations) that just prints a message:

    // Declarative //
    pipeline { (1)
        agent any // <2> (3)
        stages { (4)
            stage('Build') { (5)
                steps { (6)
                   echo 'This is a minimal pipeline.' (7)
                }
            }
        }
    }
    // Scripted //
    node { (2)
        checkout scm (3)
        stage ('Build') { (5)
            echo 'This is a minimal pipeline.' (6)
        }
    }
    1 All Declarative Pipelines start with a pipeline section.
    2 Select where to run this Pipeline, in this case "any" agent, regardless of label.
    3 Declarative automatically performs a checkout of source code on the agent, whereas Scripted Pipeline users must explicitly call checkout scm,
    4 A Declarative Pipeline is defined as a series of stages.
    5 Run the "Build" stage.
    6 Each stage in a Declarative Pipeline runs a series of steps.
    7 Run the echo step to print a message in the Console Output.
    If you are familiar with Scripted Pipeline, you can toggle the above Declarative code sample to show the Scripted equivalent.

    Once I add the Pipeline above to my Jenkinsfile and run "Branch Indexing", my Jenkins will pick it up and run run it. We see that the Declarative Pipeline has added stage called "Declarative: Checkout SCM":

    Minimal Declarative Pipeline

    This a "dynamic stage", one of several the kinds that Declarative Pipeline adds as needed for clearer reporting. In this case, it is a stage in which the Declarative Pipeline automatically checkouts out source code on the agent.

    As you can see above, we didn’t have to tell it do any of this,

    Console Output
    [Pipeline] node
    Running on osx_mbp in /Users/bitwiseman/jenkins/agents/osx_mbp/workspace/blog_add-declarative-pipeline
    [Pipeline] {
    [Pipeline] stage
    [Pipeline] { (Declarative: Checkout SCM)
    [Pipeline] checkout
    Cloning the remote Git repository
    { ... truncated 20 lines ... }
    [Pipeline] }
    [Pipeline] // stage
    [Pipeline] stage
    [Pipeline] { (Build)
    [Pipeline] echo
    This is a minimal pipeline
    [Pipeline] }
    [Pipeline] // stage
    [Pipeline] }
    [Pipeline] // node
    [Pipeline] End of Pipeline
    Finished: SUCCESS

    Declarative Pipeline syntax is a little more verbose than the equivalent Scripted Pipeline, but the added detail gives a clearer, more consistent view of what the Pipeline is supposed to do. It also gives us a structure into which we can add more configuration details about this Pipeline.

    Adding Tools to Pipeline

    The next thing we’ll add in this Pipeline is a tools section to let us use Maven. The tools section is one of several sections we can add under pipeline, which affect the configuration of the rest of the Pipeline. (We’ll look at the others, including agent, in later posts.) Each tool entry will make whatever settings changes, such as updating PATH or other environment variables, to make the named tool available in the current pipeline. It will also automatically install the named tool if that tool is configured to do so under "Managing Jenkins" → "Global Tool Configuration".

    // Declarative //
    pipeline {
        agent any
        tools { (1)
            maven 'Maven 3.3.9' (2)
            jdk 'jdk8' (3)
        }
        stages {
            stage ('Initialize') {
                steps {
                    sh '''
                        echo "PATH = ${PATH}"
                        echo "M2_HOME = ${M2_HOME}"
                    ''' (4)
                }
            }
    
            stage ('Build') {
                steps {
                    echo 'This is a minimal pipeline.'
                }
            }
        }
    }
    // Scripted Not Defined //
    1 tools section for adding tool settings.
    2 Configure this Pipeline to use the Maven version matching "Maven 3.3.9" (configured in "Managing Jenkins" → "Global Tool Configuration").
    3 Configure this Pipeline to use the Maven version matching "jdk8" (configured in "Managing Jenkins" → "Global Tool Configuration").
    4 These will show the values of PATH and M2_HOME environment variables.

    When we run this updated Pipeline the same way we ran the first, we see that the Declarative Pipeline has added another stage called "Declarative: Tool Install": In the console output, we see that during this particular stage "Maven 3.3.9" gets installed, and the PATH and M2_HOME environment variables are set:

    Declarative Pipeline with Tools Section
    Console Output
    { ... truncated lines ... }
    [Pipeline] // stage
    [Pipeline] stage
    [Pipeline] { (Declarative: Tool Install)
    [Pipeline] tool
    Unpacking https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.3.9/apache-maven-3.3.9-bin.zip
    to /Users/bitwiseman/jenkins/agents/osx_mbp/tools/hudson.tasks.Maven_MavenInstallation/Maven_3.3.9
    on osx_mbp
    [Pipeline] envVarsForTool
    [Pipeline] tool
    [Pipeline] envVarsForTool
    [Pipeline] }
    [Pipeline] // stage
    { ... }
    PATH = /Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/bin:/Users/bitwiseman/jenkins/agents/osx_mbp/tools/hudson.tasks.Maven_MavenInstallation/Maven_3.3.9/bin:...
    M2_HOME = /Users/bitwiseman/jenkins/agents/osx_mbp/tools/hudson.tasks.Maven_MavenInstallation/Maven_3.3.9
    { ... }

    Running a Maven Build

    Finally, running a Maven build is trivial. The tools section already added Maven and JDK8 to the PATH, all we need to do is call mvn install. It would be nice if I could split the build and the tests into separate stages, but Maven is famous for not liking when people do that, so I’ll leave it alone for now.

    Instead, let’s load up the results of the build using the JUnit plugin, however the version that was just built, sorry.

    // Declarative //
    pipeline {
        agent any
        tools {
            maven 'Maven 3.3.9'
            jdk 'jdk8'
        }
        stages {
            stage ('Initialize') {
                steps {
                    sh '''
                        echo "PATH = ${PATH}"
                        echo "M2_HOME = ${M2_HOME}"
                    '''
                }
            }
    
            stage ('Build') {
                steps {
                    sh 'mvn -Dmaven.test.failure.ignore=true install' (1)
                }
                post {
                    success {
                        junit 'target/surefire-reports/**/*.xml' (2)
                    }
                }
            }
        }
    }
    // Scripted //
    node {
        checkout scm
    
        String jdktool = tool name: "jdk8", type: 'hudson.model.JDK'
        def mvnHome = tool name: 'mvn'
    
        /* Set JAVA_HOME, and special PATH variables. */
        List javaEnv = [
            "PATH+MVN=${jdktool}/bin:${mvnHome}/bin",
            "M2_HOME=${mvnHome}",
            "JAVA_HOME=${jdktool}"
        ]
    
        withEnv(javaEnv) {
        stage ('Initialize') {
            sh '''
                echo "PATH = ${PATH}"
                echo "M2_HOME = ${M2_HOME}"
            '''
        }
        stage ('Build') {
            try {
                sh 'mvn -Dmaven.test.failure.ignore=true install'
            } catch (e) {
                currentBuild.result = 'FAILURE'
            }
        }
        stage ('Post') {
            if (currentBuild.result == null || currentBuild.result == 'SUCCESS') {
                junit 'target/surefire-reports/**/*.xml'  (2)
            }
        }
    }
    1 Call mvn, the version configured by the tools section will be first on the path.
    2 If the maven build succeeded, archive the JUnit test reports for display in the Jenkins web UI. We’ll discuss the post section in detail in the next blog post.
    If you are familiar with Scripted Pipeline, you can toggle the above Declarative code sample to show the Scripted equivalent.

    Below is the console output for this last revision:

    Console Output
    { ... truncated lines ... }
    + mvn install
    [INFO] Scanning for projects...
    [WARNING] The POM for org.jenkins-ci.tools:maven-hpi-plugin:jar:1.119 is missing, no dependency information available
    [WARNING] Failed to build parent project for org.jenkins-ci.plugins:junit:hpi:1.20-SNAPSHOT
    [INFO]
    [INFO] ------------------------------------------------------------------------
    [INFO] Building JUnit Plugin 1.20-SNAPSHOT
    [INFO] ------------------------------------------------------------------------
    [INFO]
    [INFO] --- maven-hpi-plugin:1.119:validate (default-validate) @ junit ---
    [INFO]
    [INFO] --- maven-enforcer-plugin:1.3.1:display-info (display-info) @ junit ---
    [INFO] Maven Version: 3.3.9
    [INFO] JDK Version: 1.8.0_92 normalized as: 1.8.0-92
    [INFO] OS Info: Arch: x86_64 Family: mac Name: mac os x Version: 10.12.3
    [INFO]
    { ... }
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESS
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time: 03:25 min
    [INFO] Finished at: 2017-02-06T22:43:41-08:00
    [INFO] Final Memory: 84M/1265M
    [INFO] ------------------------------------------------------------------------

    Conclusion

    The new Declarative syntax is a significant step forward for Jenkins Pipeline. It trades some verbosity and constraints for much greater clarity and maintainability. In the coming weeks, I’ll be adding new blog posts demonstrating various features of the Declarative syntax along with some recent Jenkins Pipeline improvements.

    About the Author
    Liam Newman
    Liam Newman

    Liam started his software career as a tester, which might explain why he’s such a fan of CI/CD and Pipeline as Code. He has spent the majority of his software engineering career implementing Continuous Integration systems at companies big and small. He is a Jenkins project contributor and an expert in Jenkins Pipeline, both Scripted and Declarative. Liam currently works as a Jenkins Evangelist at CloudBees. When not at work, he enjoys testing gravity by doing Aikido.