Recent Blog Posts

This is a guest post by Liam Newman, Technical Evangelist at CloudBees. This year’s Jenkins World conference was a huge milestone for the Jenkins project - the first global event for the Jenkins community. It brought users and contributors together to exchange ideas on the current state of the project, celebrate accomplishments of the past year, and look ahead at all the exiting enhancements coming down the pipe(line). Contributor Summit To kick off Jenkins World, we had a full day "Contributor Summit". Jenkins is a distributed project with contributors from all over the globe. Conferences like this are perfect time to get contributors together face-to-face, to talk through current issues and upcoming plans for the the project. Some key topics discussed during this summit were: Infrastructure - In the past year, the Jenkins project has moved new domain name, a statically generated website, and has entered a partnership with Microsoft to host to host infrastructure on Azure. Events - A year ago, there were five Jenkins Area Meetups, today there are 37 around the world, with ~7000 members. Security - Daniel Beck has done a great job a "Security Officer" for the project over the last year. Jenkins 2 includes tighter security out of the box, 9 security alerts have been addressed, and the Security Team is continuing to evaluate threats as they are reported. Pipeline - Pipeline has been a success and there many improvements on the way, including better Pipeline Library support, a UI-based Pipeline Editor, and Declarative Pipeline syntax. Blue Ocean - Blue Ocean announced their "1.0 Beta" release and discussed their roadmap. Storage Pluggability - One of the big upcoming goals is reducing Jenkins' dependence on local file system storage on the server system (job configuration, build logs, etc.). There was extensive discussion of how to accomplish this goal. Keynote: The State of Jenkins 2016 The next day, Kohsuke gave a great keynote, showing how far the project as come this year and where it is headed. You can get the slides here or see the full video below. What’s Next? Overall, Jenkins World was a very enjoyable event. I’m sure everyone came away having learned a lot and made many new connections. I know I’m excited to see what the coming year brings for Jenkins and the Jenkins community. Don’t forget that there are many ways to continue to build connections to the rest of the Jenkins community throughout the year, such as the Jenkins Online Meetup which hosts online events year-round. Or, see if there is a Jenkins Area Meetup (JAM) near you. If there isn’t, take a look at the Jenkins Area Meetup page to see about starting one. Thanks, and I hope to see you all and Jenkins World 2017!

This is a guest post by Liam Newman, Technical Evangelist at CloudBees. As I mentioned in my previous post, Jenkins World brought together Jenkins users from organizations of all sizes. It also brought together Jenkins users of all skill levels; from beginners to experts (including to JAM organizers, board members, and long time contributors). A number of those experts also volunteered to staff the Open Source Hub’s "Ask the Experts" desk throughout the conference to answer Jenkins questions. This included, but was not limited to: Paul Allen, R Tyler Croy, James Dumay, Jesse Glick, Eddú Meléndez Gonzales, Jon Hermansen, Owen Mehegan, Oleg Nenashev, Liam Newman, Christopher Orr, Casey Vega, Mark Waite, Dean Yu, and Keith Zantow. I actually chose to spend the majority of my time at the booth. It was fantastic to hear all the different ways people are using Jenkins and wanting use Jenkins to do even more. I answered dozens of questions on both days of the conference, often learning new things in the process of answering them. And for questions that were beyond any one person’s knowledge, there was such a breadth of expertise, very few questions were beyond our combined abilities. While "Ask the Experts" saw a lot traffic, the Open Source Hub’s lunch-time demos drew really big crowds. They covered wide range of subjects in a quick succession and offered people a chance to be introduced to new areas of in Jenkins without spending a whole session on them. Some demos were only presented at lunch while others were abbreviated versions of longer talks presented at other times during the conference. Here’s the full list with related links: Keith Zantow gave a live demo of Blue Ocean in Action on their live Jenkins instance. Christopher Orr presented a lightning version of his talk Pipelines for building and deploying Android apps ( Slides) ( video). Oleg Nenashev showed a different way to manage security with the Ownership plugin for Jenkins ( Slides). Alex Somai presented his Google Summer of Code (GSoC) 2016 project, the External Workspace Manager plugin for Jenkins Pipeline ( GSOC Video). Mark Waite discussed Git plugin - large repos, submodule authentication and more ( Slides). Liam Newman gave a live demo of Notifications with Jenkins Pipeline (based on this blog post). Jesse Glick talked about Extending Pipeline with Libraries using the Pipeline Shared Groovy Libraries Plugin Jon Hermansen demonstrated some cool ways to use Multibranch Pipelines + Git symbolic-ref to optimize build times. R Tyler Croy showed the power of Docker and Pipeline ( Slides) R Tyler Croy also showed how easy it can be to migrate from Freestyle to Pipeline ( Slides) Casey Vega gave a live demo, package.json and Jenkins, on using package.json to control all aspects of Jenkins builds. Andrew Bayer presented at lightning version of his talk, A simpler way to define Jenkins Pipelines ( Slides) ( Video). Thank you to everyone who staffed the booth and gave demos. Also, thanks to everyone who attended the demos and came by to ask questions. If you have more questions, you don’t have to wait until next year’s Jenkins World. Join the jenkinsci-users mailing list or the #jenkins IRC channel to get help from experts around the world. And finally, a special thanks to the Jenkins Events officer, Alyssa Tong, for getting the entire booth designed, prepared, and keeping everything on track before, during, and after the conference.

This is a guest post by Liam Newman, Technical Evangelist at CloudBees. One of the great features of Jenkins is how far it can scale, not only from a software perspective, but also from an organizational one. From a single Jenkins controller with one or two agents to a multiple controller with thousands of agents, from a team of only a few people to a whole company with multiple disparate departments and organizations, you’ll find space where Jenkins is used. Like any software or organization, there are common challenges for increasing scale with Jenkins and some common best practices, but there are also some unique solutions. A big conference like Jenkins World brings users from all scales together to see how people in other organizations at similar or greater scale are solving similar problems. Here’s a recap of the some of the sessions on scaling Jenkins, with links to slides and videos shared by CloudBees : Paul Miles and Grant Dickie of NPR talked about JenkinsOps: An Initiative to Streamline and Automate Jenkins . They shared ways their team has used Jenkins to automate many of the administrative tasks related to managing feature code branches, handling deployments, running tests, and properly configuring their environments. They also showed code samples and talked about future challenges in their quest to achieve continuous deployment. Download Presentation Stream Presentation Maxfield F Stewart of Riot Games showed how they built an integrated Docker solution using Jenkins in Thinking Inside the Container: A Continuous Delivery Story He showed how their system allows engineers around the company to submit Docker images as build environments. This has let their containerized farm now create over 10,000 containers per week and handles nearly 1,000 jobs at a rate of about 100 jobs per hour. And they have done this using readily available, open source Jenkins plugins. He also talked about how they settled on this design, lessons learned, best practices, and how to build and scale other similar system. Download Presentation Stream Presentation How to Do Continuous Delivery with Jenkins Pipeline, Docker and Kubernetes , presented by James Strachan of Red Hat, showed how to use Jenkins Pipeline with Docker and Kubernetes to implement a complete end-to-end continuous delivery and continuous improvement system using open source software for both microservices and monolithic applications. He demonstrated how to create or import projects, and have them automatically build, run system and integration tests, stage, and finally deploy. He also showed to manage and update those deployed applications using continuous delivery practices. Download Presentation Watch the Video Demo from the Presentation Carlos Sanchez of CloudBees discussed Scaling Jenkins with Docker: Swarm, Kubernetes or Mesos? He compared various Docker Swarm, Apache Mesos, and Kubernetes in terms of their ability to dynamically scale in Jenkins by running jobs inside containers. He also discussed the pros and cons, best practices, level of Jenkins support for each of these technologies. Download Presentation Stream Presentation Stephen Connolly of CloudBees asked " So, You Want to Build the World’s Biggest Jenkins Cluster? " and explained how to do so. He started with real world results realized by Jenkins users who have built large clusters. Next, he showed experiments around scaling some individual sub-components of Jenkins in isolation to see what challenges have been faced when integrated. Finally, he arrived at recipes for building Jenkins clusters with different scaling capabilities and making existing Jenkins clusters more efficient. Download Presentation Stream Presentation Bill Houston and Ali Raza of Splunk gave a talk in two parts, Jenkins at Splunk and Splunking Jenkins In the first part, Bill showed how Splunk uses Jenkins to implement their end-to-end CI system. They discussed features and design goals, challenges they encountered, and how they addressed these challenges. In the second part, Ali showed how to use the Jenkins Splunk plugin. Using plugin, he gathered test results and Jenkins environment data, and delivered it to a Splunk indexer for analysis and presentation. Download Presentation Stream Presentation David Hoover of Google talked about Jenkins inside Google . Last year, they presented their initial investigations and stress testing as they prepared to deploy a large-scale Jenkins installation at Google. Now, with a year of real-world use under their belts, they returned to present on how their expectations held up, what new issues they encountered, how they have addressed those issues, and the challenges and opportunities they see ahead. Download Presentation Stream Presentation

This is a guest post by Liam Newman, Technical Evangelist at CloudBees. As someone who has managed Jenkins for years and manually managed jobs, I think pipeline is fantastic. I spent much of the conference manning the Ask the Experts desk of the "Open Source Hub" and was glad to find I was not alone in that sentiment. The questions were not "Why should I use Pipeline?", but "How do I do this in Pipeline?" Everyone was interested in showing what they have been able to accomplish, learning about best practices, and seeing what new features were on the horizon. The sessions and demos on Pipeline that I saw were all well attended, but no one could have seen all of them. Here’s a recap of the some of the sessions on Jenkins Pipeline, with links to slides and videos shared by CloudBees : Jesse Glick discussed the past, present, and future of Jenkins Pipeline in Directions for Pipeline . He reviewed a broad range of improvements made to Pipeline over the last year, including syntax, documentation, plugin support, and stability. He reviewed the changes currently underway. He also pointed out that many of the improvements have been driven by user feedback and invited everyone to continue to participate in making pipeline even better. Download Presentation Stream Presentation In Pipelining DevOps with Jenkins and AWS , Jimmy Ray of nextSource showed how Pipeline can be used to automate CI/CD build processes, and how to integrate Jenkins and Pipeline with AWS. He also discussed some admin-level considerations, such as how to install Jenkins on EC2 and the merits of "LTS" and "latest build". Download Presentation Stream Presentation Christopher Orr examined how to create " Continuous Build and Delivery Pipelines for Android " applications. He showed how to set up Android-capable build agents, ensure traceable application releases, reporting warnings, run various types of tests, and deploy and app to Google Play. This included live demonstrations and discussion of best practices. Download Presentation Stream Presentation Andrew Bayer presented A New Way to Define Jenkins Pipelines . He showed the next evolution for Pipeline, based on a simpler declarative model. This declarative syntax for Pipeline still supports the creation of complex pipelines, including complete build environments, post-build actions, and notifications, while also being easier to understand. This declarative syntax also makes in it easier to implement other interesting scenarios such as early validation of pipelines and a visual pipeline editor. Download Presentation Stream Presentation In Perfecting Your Development Tools: Updates to the Helix Plugin for Jenkins , Paul Allen of Perforce walked through using Perforce’s "Monorepo" model with Jenkins Pipeline. He explained in detail how to work with the Perforce"P4" plugin in Jenkins, including credential passing and workspace management. Of particular interest was his side-by-side comparison the various actions done with the Jenkins UI vs Pipeline. Download Presentation Sam Van Oort demonstrated strategies for faster pipelines in The Need For Speed: Building Pipelines To Be Faster . He discussed various elements that contribute to making pipelines faster or slower, such a number of resources and latency. He then showed several best practices for constructing pipelines that have lower turnaround times and reduced resource use. He also reviewed plugins and tools that can help analyze and visualize pipeline performance, including the Pipeline Stage View plugin and Blue Ocean. Download Presentation Stream Presentation Bobby Sandell and James T. Nord talked about what not to do with Pipeline in No, You Shouldn’t Do That! Lessons from Using Pipeline . They told the story of their own experiences as early adopters of Jenkins Pipeline at CloudBees. They described a number of key scenarios they attempted to address, detailed various mistakes and false starts, and finally share what they learned in each case. Download Presentation Stream Presentation Alexandru Somai gave a lightning talk on his Google Summer of Code (GSoC) 2016 project, " External Workspace Manager Plugin for Jenkins Pipeline". The build workspace for Jenkins projects may become very large. Alex showed how the External Workspace Manager plugin addresses this issue, adding support for managing and reusing the same workspace between multiple pipeline builds. A recording of his presentation for GSOC is available here. How to Do Continuous Delivery with Jenkins Pipeline, Docker and Kubernetes , presented by James Strachan of Red Hat, showed how to use Jenkins Pipeline with Docker and Kubernetes to implement a complete end-to-end continuous delivery and continuous improvement system using open source software for both microservices and monolithic applications. He demonstrated how to create or import projects, and have them automatically build, run system and integration tests, stage, and finally deploy. He also showed to manage and update those deployed applications using continuous delivery practices. Download Presentation Stream Demo Video

This is a guest post by Liam Newman, Technical Evangelist at CloudBees. That’s a Wrap! Any way you look at it, last week’s Jenkins World Conference 2016 was a huge success. In 2011, a few hundred users gathered in San Francisco for the first "Jenkins User Conference". Over successive years, this grew into several yearly regional Jenkins user conferences. This year, over 1,300 people came from around the world to "Jenkins World 2016", the first global event for the Jenkins community. This year’s Jenkins World conference included: Keynote presentation by Jenkins creator, Kohsuke Kawaguchi, announcing a number of great new Jenkins project features, such as "Blue Ocean". More than 50 sessions on everything from the new "Blue Ocean" UI, to "Continuous Security" to "Dockerizing Jenkins". Jenkins Open-source Hub, with "Ask the Experts" and demos by 20+ Jenkins contributors. Booths from 30+ sponsors. Stickers! Over the next week, I’ll be posting highlights from the event, including slides, videos, and links to other useful resources. Stay tuned!

On September 6th we had a Jenkins Online Meetup. This meetup was the second event in the series of Plugin Development meet ups. At this meetup we were talking about Jenkins Web UI development. Talks 1) Classic Jenkins UI framework - Daniel Beck In the first part of his talk, Daniel presented how Stapler, the web framework used in Jenkins, works, and how you can add to the set of URLs handled by Jenkins. In the second part he was talking about creating new views using Jelly and Groovy, and how to add new content to existing views. Keywords: Stapler, Jelly, Groovy-defined UIs 2) Developing modern Jenkins UIs with Javascript - Tom Fennelly Feel that Jenkins UI is a bit old? You are not alone. In addition to the old stack Jenkins offers a framework for writing UI components in Javascript with help of Node.js. Tom presented this new engine, which is being used in new Jenkins Web UI components like Jenkins installation wizard. He also provided several examples from the BlueOcean project he is working on. Keywords: Node.js, ReactJS, Jenkins JS Builder, Jenkins Design Language, Blue Ocean Links Meetup page Event page: Plugin Development. Web UI Webinar recording Want to conduct a meetup? We are looking for speakers, who would be interested to share their experience about Jenkins best-practices, war stories and plugin development. If you are interested to conduct a presentation, please contact meetup organizers using meetup.com “contact organizers” feature or via the events@lists.jenkins-ci.org mailing list.

This is a guest post by Jenkins World speaker R Tyler Croy, infrastructure maintainer for the Jenkins project. I don’t think I have ever met a tools, infrastructure, or operations team that did not have a ton of work to do. The Jenkins project’s infrastructure"team" is no different; too much work, not enough time. In lieu of hiring more people, which isn’t always an option, I have found heavy automation and continuous delivery pipelines to be two solutions within reach of the over-worked infrastructure team. As a big believer in the concept of "Infrastructure as Code", I have been, slowly but surely, moving the project’s infrastructure from manual tasks to code, whether implemented in our Puppet code-base, Docker containers, or even as machine specifications with Packer. The more of our infrastructure that is code, the more we can apply continuous delivery practices to consistently and reliably build, test and deliver our infrastructure. This approach integrates nicely with Jenkins Pipeline, allowing us to also define our continuous delivery pipelines themselves as code. For example, by sanity-checking our BIND zone files: Jenkinsfile node('docker') { def dockerImage = 'rtyler/jenkins-infra-builder' checkout scm docker.image(dockerImage).inside { sh "/usr/sbin/named-checkzone jenkins-ci.org dist/profile/files/bind/jenkins-ci.org.zone" sh "/usr/sbin/named-checkzone jenkins.io dist/profile/files/bind/jenkins.io.zone" } } Or delivering our Docker containers automatically to Docker Hub, with a Jenkinsfile such as: Jenkinsfile node('docker') { checkout scm /* Get our abbreviated SHA-1 to uniquely identify this build */ def shortCommit = sh(script: 'git rev-parse HEAD', returnStdout: true).take(6) stage 'Build ircbot' { withEnv(["JAVA_HOME=${tool 'jdk8'}", "PATH+MVN=${tool 'mvn'}/bin"]) } sh 'make bot' } } def whale stage 'Build container' { whale = docker.build("jenkinsciinfra/ircbot:build${shortCommit}") } stage 'Deploy container' { /* Push to Docker Hub */ whale.push() } } In my talk at Jenkins World (September 14th, 3:00 - 3:45pm in Exhibit Hall A-1) I will discuss these Jenkinsfiles along with some of the strategies, patterns and code used with the Jenkins project’s open source infrastructure to get the most out of the team’s limited time. R Tyler will be presenting more about continous delivery of infrastructure at Jenkins World in September. Register with the code JWFOSS for 20% off your full conference pass.

This is a guest post by R. Tyler Croy, who is a long-time contributor to Jenkins and the primary contact for Jenkins project infrastructure. He is also a Jenkins Evangelist at CloudBees, Inc. I have been heavily using Jenkins Pipeline for just about every Jenkins-related project I have contributed to over the past year. Whether I am building and publishing Docker containers, testing infrastructure code or publishing this very web site, I have been adding a Jenkinsfile to nearly every Git repository I touch. Implementing Pipeline has been rewarding, but has not been without its own challenges. That’s why I’m excited to see lots of different Jenkins Pipeline related content in the agenda at Jenkins World. I don’t think it’s possible for a single person to attend all of the Pipeline talks, or the Pipeline-related demos in the "Open Source Hub", but fortunately CloudBees will be recording the sessions. If you have Pipeline-related questions unanswered by all these presentations, feel free to join us at the "Open Source Hub" in the expo hall and Ask the Experts. On the first day of Jenkins World (September 13th), Isaac Cohen is hosting a workshop titled Let’s Build a Jenkins Pipeline which may be interesting to you if you haven’t yet worked with Pipeline. Pipelining DevOps with Jenkins and AWS September 14th 2:00 PM - 2:45 PM, Exhibit Hall A-1 Automated workflow is a proven method for removing process variability. DevOps pipelines are the next step in the evolution of CI/CD/DevOps. This talk covers Jenkins pipelines, both with and without AWS integration, and explains how Jenkins can be used to create, execute and manage pipelines. — Jimmy Ray of nextSource Perfecting Your Development Tools: Updates to the Helix Plugin for Jenkins September 14th 5:00 PM - 6:00 PM, Exhibit Hall C Considering a mono repo that can manage all your source code, binary and other assets? Join us at the Perforce Birds of a Feather Session for updates and discussions around the Helix Plugin for Jenkins (or ‘P4 plugin’). This session will look at the latest DSL PipeLine support in the ‘P4 plugin’ for Jenkins and will include a live demo. We will show you how to map your Branches and Streams into a Jenkins Workspace, publish assets back into Helix, and more. You may even get a sneak preview at the latest ‘P4 plugin’ for Jenkins that allows you the freedom to query and run commands from within Jenkins directly against your Helix connection. — Paul Allen of Perforce Continuously Deploying Containers with Jenkins Pipeline to Docker Swarm Cluster September 14th 3:00 PM - 3:45 PM, Exhibit Hall A-3 Many of us have already experimented with Docker - for example, by running one of the pre-built images from Docker Hub. It is possible that your team might have recognized the benefits that Docker provides in building microservices and the advantages the technology could bring to development, testing, integration and, ultimately, production. However, you must create a comprehensive build pipeline before deploying any containers into a live environment. Integrating containers into a CD pipeline is far from easy. Along with the benefits Docker brings, there are challenges both technically and process-related. This presentation attempts to outline the steps you need to take for a fully-automated Jenkins pipeline that continuously builds, tests and deploys microservices into a Docker Swarm cluster. — Viktor Farcic No, You Shouldn’t Do That! Lessons from Using Pipeline September 15th 10:30 AM - 11:15 AM, Exhibit Hall A-1 Pipeline is as powerful as a loaded gun, but with skill can be as delicate as a surgeon’s knife. This talk will give an overview of health and safety so that you can avoid shooting yourself in the head and walk the path to medical school. It will cover not only what not to do, but also why, and share some solutions so you are not left high and dry. Both James and Bobby have bullet wounds from “Champagning” pipeline to automate the test and release of several of the CloudBees products and can occasionally still be seen walking with a limp from shooting for the moon and hitting their feet. — Bobby Sandell and James T. Nord of CloudBees Docker Image Lifecycle Implemented with Jenkins Pipeline September 15th 11:30 AM - 12:15 PM, Exhibit Hall A-2 While Docker has enabled an unprecedented velocity of software production, it is all too easy to spin out of control. A promotion-based model is required to control and track the flow of Docker images as much as it is required for a traditional software development lifecycle. We will demonstrate how to go from development to containerization to distribution utilizing binary management promotion in a framework implemented on Jenkins, using the Pipeline functionality. — Mark Galpin Directions for Pipeline September 15th 11:30 AM - 12:15 PM, Exhibit Hall A-1 The Pipeline feature has matured and is now included in Jenkins 2.0. During the time since its release, copious user feedback has been received about missing features and pain points. Come hear about some things we know should be worked on - or are already in progress - and bring your suggestions. — Jesse Glick of CloudBees How to Do Continuous Delivery with Jenkins Pipeline, Docker and Kubernetes September 15th 2:30 PM - 3:15 PM, Great America J In this talk, we’ll show how to use Jenkins Pipeline together with Docker and Kubernetes to implement a complete end-to-end continuous delivery and continuous improvement system for microservices and monolithic applications using open source software. We’ll demonstrate how to easily create new microservices projects or import existing projects, have them automatically built, system and integration tested, staged and then deployed. Once deployed, we will also see how to manage and update applications using continuous delivery practices along with integrated ChatOps - all completely automated! — James Strachan of Red Hat Introducing a New Way to Define Jenkins Pipelines September 15th 3:45 PM - 4:30 PM, Great America J Pipeline is quickly establishing itself as the direction that Jenkins jobs are going, enabling the definition of a complete CD pipeline in a single job; Pipeline as Code via the “Jenkinsfile”; job durability across controller restarts; and more. I’ll be talking here about the next evolution for Pipeline: a simple, declarative model to define your Pipelines with no need to write scripts. This configuration syntax for Pipeline allows you to automatically configure all stages of your pipeline, the complete build environment, post-build actions, notifications and more. All while providing syntactic and semantic validation before the build actually gets going. — Andrew Bayer of CloudBees The Need For Speed: Building Pipelines To Be Faster September 15th 4:45 PM - 5:30 PM, Exhibit Hall A-1 Response time is paramount for a CI/CD system. In this session, you will see how a few best practices in constructing pipelines can yield faster turnaround times and reduced resource use. We’ll also run through plugins and tools to analyze and visualize performance, including the Pipeline Stage View plugin. If time permits, we may briefly discuss some of the computer science theory behind different aspects of performance. — Sam Van Oort of CloudBees Continuously Delivering Continuous Delivery Pipelines September 15th 4:45 PM - 5:30 PM, Exhibit Hall J Our 600-person IT organization has committed to implementing continuous delivery practices enterprise-wide. This isn’t a single momentous event put in place overnight. Rather, it’s a strategic journey towards a common goal, and through which each application will take its own unique path. A seminal component of our CD journey is the Pipeline plugin and it has become our standard for CD pipeline orchestration. We will discuss a few of the diverse paths taken by the application teams at our company and show how the use of the Pipeline plugin has uniquely enabled continuous delivery for us in a way that no competing tool can. — Neil Hunt of Aquilent CD Pipelines as Code with Github and Bitbucket September 15th 4:45 PM - 5:30 PM, Exhibit Hall J Pipeline Multibranch projects come as a natural evolution of pipeline as code: define your CD pipeline in your source code repository and Jenkins will create isolated branch and pull requests jobs for it. This talk is about the integration of the Pipeline Multibranch plugin with Github and Bitbucket as branch sources. — Antonio Muñiz of CloudBees Register for Jenkins World in September with the code JWFOSS for a 20% discount off your pass.