We’re counting down the weeks until Blue Ocean 1.0, which is planned for the end of March. If you hadn’t picked up on the hint in my previous post, most of the Blue Ocean development team is in Australia, where it is currently the middle of summer. As I write this it is about 1000 degrees outside. Emergency measures such as air-conditioning and beer have been deployed in order to continue Blue Ocean development. This week featured a new beta with the SCM API changes; many bug fixes, and some version bumps went out in beta 22. We also got some fresh new designs coming soon, though not in time for beta 22. Some development highlights: Beta 22 went out featuring the new SCM API with better use of GitHub API rate limits. A fix for publishing of Server Side Events that made one CPU spin up to 100% was fixed (not good unless you want to heat up your room) Some new refinements to the design merged to the master branch (see images below). Beta 22 featured the 1.0 version of Declarative Pipeline An Australian translation was added; really critical stuff, I know.. The Acceptance Test Harness (ATH) was stabilised a bit and it now covers creating Pipelines from Git, which we talked about in late January. The Visual Pipeline Editor was released to the main Update Center as a preview release, ready to play with! Some small performance improvements I’m looking forward to those fancy new designs making their way into an upcoming release too. Lovely! Hopefully you see more green than I do…​ Anyways, up next for Blue Ocean: Creation of Pipelines from GitHub, including auto-discovery of new Pipelines. Closer to a "release candidate" Working on filtering the activity view for "per branch" views Better reporting of durations of stages, steps, and runs Enjoy! If you’re interested in helping to make Blue Ocean a great user experience for Jenkins, please join the Blue Ocean development team on Gitter!

With only a couple of months left before Blue Ocean 1.0, which is planned for the end of March, I have been highlighting some of the good work being finished up by the developers hacking on Blue Ocean. This week was a grab bag of important behind-the-scenes features and finalising the preview of the editor. The merge of the SCM API changes also made it in. The editor has the new sheets style of editing (there will be blogs and more on this in the next few weeks): Some highlights: Fix to async loading of resources like translations, so screens don’t "flash" when they are loaded (i18n improvement) Links in notifications can be configured to point to classic or Blue Ocean screens Time reporting works better when browser clock is out of sync with server SECURITY-380 was backported into a small fix for those that aren’t running the latest LTS (but you should ideally be running it) SCM API changes finally landed - this will be in beta 22 which should hit the update centers soon. This should make things work better with GitHub rate limits. Beta 21 was released The editor reached "preview" release state ready for use with the newly announced Declarative Pipeline stuff. Also, a reference to Australian pop culture had to be removed, sadly. Up Next: Some cosmetic changes around headers to make it much nicer and clearer Favorite improvements GitHub Org-based Pipeline creation Editor available in the general update center Beta 22 with SCM improvements and no more GitHub rate limit hassles Many fixes Improvements to the Acceptance Test Harness to reduce the number of false-positives. Enjoy! If you’re interested in helping to make Blue Ocean a great user experience for Jenkins, please join the Blue Ocean development team on Gitter!

As we get closer to Blue Ocean 1.0, which is planned for the end of March, I have started highlighting some of the good stuff that has been going on. This week was 10 steps forward, and about 1.5 backwards…​ There were two releases this week, b19 and b20. Unfortunately, b20 had to be released shortly after b19 hit the Update Center as an incompatible API change in a 3rd party plugin was discovered. Regardless, the latest b20 has a lot of important improvements, and some very nice new features. A first cut of the "Create Pipeline" UX, seen above, allowing you to create Git based Multibranch Pipelines like you have never seen before. Handling network disconnections from the browser to server (eg server restart, network etc) gracefully with a nice UI. More precise time information for steps and running Pipelines. More information when a Pipeline is blocked on infrastructure, such as when the Pipeline is waiting for an agent to become available. Fixed a really embarrassing typo (a prize if you spot it). Test reports now include stdout and stderr Better support for parallel visualisation, such as when a parallel step exists outside of a stage. The Visual Editor also had another release, with the "sheets" visual component and better validation. Creation Currently this is hidden behind a feature toggle, to access append?blueCreate to the URL in you browser, and then press the "New Pipeline" button. Currently it lets you quickly create a Pipeline from Git, add credentials, etc, in a very nice UX. More SCM types are being added to support this. Reconnect/disconnect As Blue Ocean is a very "live" style of UX, if your network becomes unavailable, or the server is restarted, it is good to know in case you were staring at the screen waiting for something to happen (don’t you have anything better to do??). When this happens, now you get a polite message, and then when the connection is restored, even if you are waiting for a Pipeline run to finish, it will then notice this, and refresh things for you: Note the opacity changes to make it clear even if you don’t see the little message. Very nice addition for those of us who work on a train far to often. Up next What is up next: SCM Api changes should land, making things much better for users of GitHub, Bitbucket, and many more. Creating Pipelines from GitHub (including automatic discovery). Lots of fixes and enhancements in the Pipeline from all over the place More ATH [ 1 ] coverage against regressions More Visual Editor releases as Declarative Pipeline reaches version 1.0 Improvements to i18n There was also a couple of "alternative beta" releases in the "Experimental Update Center" to help test the new SCM API improvements for better use of GitHub APIs (based on this branch) I do not recommend trying this branch unless you know what you are doing, as this will migrate some data, but help testing it would be appreciated! Enjoy! If you’re interested in helping to make Blue Ocean a great user experience for Jenkins, please join the Blue Ocean development team on Gitter! 1. Acceptance Test Harness

As we get closer to Blue Ocean 1.0, which is planned for the end of March, I have started highlighting some of the good stuff that has been going on, and this week was a very busy week. A new Blue Ocean beta ( b18) was released with: Parametrized pipelines are now supported! i18n improvements Better support for matrix and the evil (yet somehow still used) Maven project type (don’t use it!) SSE fixes for IE and Edge browsers An alpha release of the Visual Editor for Jenkinsfiles on top of Declarative Pipeline has snuck into the "experimental" update center. Andrew will be talking about Declarative Pipelines at FOSDEM next week. Parametrized Pipelines You would know this if you followed Thorsten’s twitter account. That twitter account is mostly pics of Thorsten in running gear, but occasionally he announces new features as they land. When you run a pipeline that requires parameters, it will popup a dialog like this no matter where you run it from. Most input types are supported (similar to input), with a planned extension point for custom input types. Editor A very-very early version of the Blue Ocean Pipeline Editor plugin that will set your hair on fire of the editor is in the experimental update center. Declarative pipelines are still not at version 1.0 status, but will be shortly. This editor allows you to roundtrip Jenkinsfiles written in this way, so they can be edited as text, or visually. The steps available are discovered form the installed plugins. One to watch. So, what’s next? Creation of Git Pipelines, and likely GitHub too. Show parallel branches that aren’t in a stage visually Show stderr/out in test reports Show more information when Jenkins is "busy", such as when agents are coming online, in the Pipeline view Enjoy! If you’re interested in helping to make Blue Ocean a great user experience for Jenkins, please join the Blue Ocean development team on Gitter!

As we get closer to Blue Ocean 1.0, which is planned for the end of March, I figured it would be great to highlight some of the good stuff that has been going on. It’s been a busy-as-usual week as everyone comes back from vacation. A couple of new betas went out this week. Of note: input to Pipelines is now supported, a much asked for feature (see below) A new French translation Some optimisations (especially around reducing number of HTTP calls). We have started using gtmetrix.com to measure changes on"dogfood" to get some numbers around optimisations on the web tier. And a grab bag of other great bug fixes. Also a bunch of work has been done to support parametrized pipelines, as well as creation of new multibranch pipelines (both are much asked for). There is also now an "official" Docker image being published to Docker Hub. The Pipeline building the container is run weekly and will be picking up newly tagged releases of Blue Ocean. Running the latest can be as simple as: docker run -p 8888:8080 jenkinsci/blueocean:latest This is built on the incredibly popular official "jenkins" image (10M pulls can’t all be wrong!). The container also has tags available (e.g. jenkinsci/blueocean:1.0.0-b16) for grabbing a specific released version. Up next for Blue Ocean development as we march towards 1.0: Support for parametrized jobs. For which a bunch of api work has already been done. Creation of the new Pipeline GUI Preview release of the Visual Editor for Declarative Pipeline. The new header design will be applied Enjoy! If you’re interested in helping to make Blue Ocean a great user experience for Jenkins, please join the Blue Ocean development team on Gitter!

This is a guest post by Michael Neale, long time open source developer and contributor to the Blue Ocean project. If you are running parts of your pipeline on Linux, possibly the easiest way to get a clean reusable environment is to use: CloudBees Docker Pipeline plugin. In this short post I wanted to show how you can avoid installing stuff on the agents, and have per project, or even per branch, customized build environments. Your environment, as well as your pipeline is defined and versioned alongside your code. I wanted to use the Blue Ocean project as an example of a project that uses the CloudBees Docker Pipeline plugin. Environment and Pipeline for JavaScript components The Blue Ocean project has a few moving parts, one of which is called the "Jenkins Design Language". This is a grab bag of re-usable CSS, HTML, style rules, icons and JavaScript components (using React.js) that provide the look and feel for Blue Ocean. JavaScript and Web Development being what it is in 2016, many utilities are need to assemble a web app. This includes npm and all that it needs, less.js to convert Less to CSS, Babel to "transpile" versions of JavaScript to other types of JavaScript (don’t ask) and more. We could spend time installling nodejs/npm on the agents, but why not just use the official off the shelf docker image from Docker Hub? The only thing that has to be installed and run on the build agents is the Jenkins agent, and a docker daemon. A simple pipeline using this approach would be: node { stage "Prepare environment" checkout scm docker.image('node').inside { stage "Checkout and build deps" sh "npm install" stage "Test and validate" sh "npm install gulp-cli && ./node_modules/.bin/gulp" } } This uses the stock "official" Node.js image from the Docker Hub, but doesn’t let us customize much about the environment. Customising the environment, without installing bits on the agent Being the forward looking and lazy person that I am, I didn’t want to have to go and fish around for a Docker image every time a developer wanted something special installed. Instead, I put a Dockerfile in the root of the repo, alongside the Jenkinsfile : The contents of the Dockerfile can then define the exact environment needed to build the project. Sure enough, shortly after this, someone came along saying they wanted to use Flow from Facebook (A typechecker for JavaScript). This required an additional native component to work (via apt-get install). This was achieved via a pull request to both the Jenkinsfile and the Dockerfile at the same time. So now our environment is defined by a Dockerfile with the following contents: # Lets not just use any old version but pick one FROM node:5.11.1 # This is needed for flow, and the weirdos that built it in ocaml: RUN apt-get update && apt-get install -y libelf1 RUN useradd jenkins --shell /bin/bash --create-home USER jenkins The Jenkinsfile pipeline now has the following contents: node { stage "Prepare environment" checkout scm def environment = docker.build 'cloudbees-node' environment.inside { stage "Checkout and build deps" sh "npm install" stage "Validate types" sh "./node_modules/.bin/flow" stage "Test and validate" sh "npm install gulp-cli && ./node_modules/.bin/gulp" junit 'reports/**/*.xml' } stage "Cleanup" deleteDir() } Even hip JavaScript tools can emit that weird XML format that test reporters can use, e.g. the junit result archiver. The main change is that we have docker.build being called to produce the environment which is then used. Running docker build is essentially a "no-op" if the image has already been built on the agent before. What’s it like to drive? Well, using Blue Ocean, to build Blue Ocean, yields a pipeline that visually looks like this (a recent run I screen capped): This creates a pipeline that developers can tweak on a pull-request basis, along with any changes to the environment needed to support it, without having to install any packages on the agent. Why not use docker commands directly? You could of course just use shell commands to do things with Docker directly, however, Jenkins Pipeline keeps track of Docker images used in a Dockerfile via the "Docker Fingerprints" link (which is good, should that image need to change due to a security patch). Links The project used as as an example is here The pipeline is defined by the Jenkinsfile The environment is defined by the Dockerfile Read more on Docker Pipeline

In this post I will show how you can make your own DSL extensions and distribute them as a plugin, using Pipeline Script. A quick refresher Pipeline has a well kept secret: the ability to add your own DSL elements. Pipeline is itself a DSL, but you can extend it. There are 2 main reasons I can think you may want to do this: You want to reduce boilerplate by encapsulating common snippets/things you do in one DSL statement. You want to provide a DSL that provides a prescriptive way that your builds work - uniform across your organisations Jenkinsfiles. A DSL could look as simple as acmeBuild { script = "./bin/ci" environment = "nginx" team = "evil-devs" deployBranch = "production" } This could be the entirety of your Jenkinsfile! In this "simple" example, it could actually be doing a multi stage build with retries, in a specified docker container, that deploys only from the production branch. Detailed notifications are sent to the right team on important events (as defined by your org). Traditionally this is done via the global library. You take a snippet of DSL you want to want to make into a DSL, and drop it in the git repo that is baked into Jenkins. A great trivial example is this: jenkinsPlugin { name = 'git' } Which is enabled by git pushing the following into vars/jenkinsPlugin.groovy The name of the file is the name of the DSL expression you use in the Jenkinsfile def call(body) { def config = [:] body.resolveStrategy = Closure.DELEGATE_FIRST body.delegate = config body() // This is where the magic happens - put your pipeline snippets in here, get variables from config. node { git url: "https://github.com/jenkinsci/${config.name}-plugin.git" sh "mvn install" mail to: "...", subject: "${config.name} plugin build", body: "..." } } You can imagine many more pipelines, or even archetypes/templates of pipelines you could do in this way, providing a really easy Jenkinsfile syntax for your users. Making it a plugin Using the global DSL library is a handy thing if you have a single Jenkins, or want to keep the DSLs local to a Jenkins instance. But what if you want to distribute it around your org, or, perhaps it is general purpose enough you want to share it with the world? Well this is possible, by wrapping it in a plugin. You use the same pipeline snippet tricks you use in the global lib, but put it in the dsl directory of a plugin. My simple build plugin shows how it is done. To make your own plugin: Create a new plugin project, either fork the simple build one, or add a dependency to it in your pom.xml / build.gradle file Put your dsl in the resources directory in a similar fashion to this (note the "package dsl" declaration at the top) Create the equivalent extension that just points to the DSL by name like this This is mostly "boiler plate" but it tells Jenkins there is a GlobalVariable extension available when Pipelines run. Deploy it to an Jenkins Update Center to share with your org, or everyone! The advantage of delivering this DSL as a plugin is that it has a version (you can also put tests in there), and distributable just like any other plugin. For the more advanced, Andrew Bayer has a Simple Travis Runner plugin that interprets and runs travis.yml files which is also implemented in pipeline. So, approximately, you can build plugins for pipeline that extend pipeline, in pipeline script (with a teeny bit of boiler plate). Enjoy!