Recent Blog Posts

This is a guest post by Jenkins World speaker Andrew Bayer, Jenkins developer at CloudBees. Over the last couple years, Pipeline as code has very much become the future of Jenkins - in fact, at this point, I’d say it’s pretty well established as the present of Jenkins. But that doesn’t mean it’s done, let alone that it’s perfect. While many developers enjoy the power and control that they get from writing Pipelines using scripting, not everyone feels the same way. A lot of developers want to specify their build as configuration and get on with building software. Pipeline scripts haven’t been a good way to do that…​until now. With new changes to Jenkins Pipeline, you are now able to define your Pipeline from configuration in your Jenkinsfile by installing the new Pipeline Model Definition plugin. It’s available today for you to try via the update center. Be sure to check the documentation for examples on how to get started for a variety of languages and platforms. Here’s a quick example based on the plugin’s own Jenkinsfile : pipeline { // Make sure that the tools we need are installed and on the path. tools { maven "Maven 3.3.9" jdk "Oracle JDK 8u40" } // Run on any executor. agent label:"" // The order that sections are specified doesn't matter - this will still be run // after the stages, even though it's specified before the stages. postBuild { // No matter what the build status is, run these steps. There are other conditions // available as well, such as "success", "failed", "unstable", and "changed". always { archive "target/**/*" junit 'target/surefire-reports/*.xml' } } stages { // While there's only one stage here, you can specify as many stages as you like! stage("build") { sh 'mvn clean install -Dmaven.test.failure.ignore=true' } } } It’s still early days for this feature, with a lot of further functionality planned to make it easier and more intuitive to define your Pipelines. All of that functionality lives on top of Pipeline scripting, so we’ll also keep improving Pipeline steps and syntax outside of the model! And perhaps most exciting, the Pipeline model will be used by an in-the-works visual editor that will be part of the Blue Ocean project - while the editor isn’t ready yet, the Pipeline Model Definition plugin will be bundled with the Blue Ocean beta for you to try out. I’ll be going into all of this and more at my talk on Thursday, September 15th, at 3:45pm at Jenkins World, and showing off the same day at the lunchtime demo theater. I can’t wait to see you all there and hear what you think of all this! Andrew will be presenting more of this concept at Jenkins World in September. Register with the code JWFOSS for 20% off your full conference pass.

At previous Jenkins User Conferences we have hosted "Contributor Summits" to gather developers and power-users in one room to discuss specific areas of Jenkins, such as Scalability, Pipeline, etc. As part of this year’s Jenkins World we’re hosting another Contributor Summit, to discuss: Blue Ocean , Pipeline and Storage Pluggability. Contributors to these three areas of the Jenkins ecosystem will be in attendance to present details of their design, requirements, and tentative roadmaps. After the presentations, the afternoon will be "unconference style" which is much more fluid to allow discussions, feedback, and brain-storming around the three focus areas. The program for the Jenkins World Contributor Summit includes: Updates from the various project officers. A discussion of the Blue Ocean technology stack, overall architecture, and how to develop plugins that integrate with Blue Ocean. Led by Keith Zantow. Presentation on the current status of Pipeline, lessons learned, new changes and the future. Led by Jesse Glick. Overview of "Storage Pluggability", a new scalability-oriented project to revamp the underlying storage mechanisms in Jenkins. Led by Kohsuke Kawaguchi. I cannot recommend participating in the Contributor Summit enough. I have found previous Summits to be immensely useful for sharing my own thoughts, as well as for hearing new perspectives from the others in attendance. Our space is limited however! I encourage you to join us, so please RSVP soon! Register for Jenkins World in September with the code JWFOSS for a 20% discount off your pass.

This is a guest post by R. Tyler Croy, who is a long-time contributor to Jenkins and the primary contact for Jenkins project infrastructure. He is also a Jenkins Evangelist at CloudBees, Inc. I find the topic of "scaling Jenkins" to be incredibly interesting because, more often than not, scaling Jenkins isn’t just about scaling a single instance but rather scaling an organization and its continuous delivery processes. In many cases when people talk about "scaling Jenkins" they’re talking about "Jenkins as a Service" or "Continuous Delivery as a Service" which introduces a much broader scope, and also more organization-specific requirements, to the problem. One of my favorite parts of a big conference like Jenkins World is getting to see how other people are solving similar problems at different organizations, in essence: " how the sausage is made." This year’s Jenkins World will be no different, with a number of sessions by developers and engineers from the companies leading the way, scaling continuous delivery and Jenkins. Register for Jenkins World in September with the code JWFOSS for a 20% discount off your pass. In the realm of "scaling Jenkins" the following sessions stand-out to me as "must-attend" for those interested in the space: JenkinsOps: An Initiative to Streamline and Automate Jenkins September 14th 4:15 PM - 5:00 PM, Exhibit Hall A-1 NPR’s Digital Media team uses Jenkins to build, test and deploy code to various staging and production environments. As the complexity of the software components, environments and tests have grown - both generally and due to our quest to achieve continuous deployment - management of Jenkins has become a challenge. In this talk, we share information about our “JenkinsOps” effort which has allowed us to automate many of the administrative tasks necessary to manage feature code branches, handle deployments, run tests and configure our environments properly. — Paul Miles and Grant Dickie of NPR Thinking Inside the Container: A Continuous Delivery Story September 15th 1:30 PM - 2:15 PM, Exhibit Hall C At Riot Games, we build a lot of software. Come learn how we built an integrated Docker solution using Jenkins that accepts Docker images submitted as build environments by engineers around the company. Our containerized farm now creates over 10,000 containers per week and handles nearly 1,000 jobs at a rate of about 100 jobs per hour. All this is done with readily available, open source Jenkins plugins. We’ll explore lessons learned, best practices and how to scale and build your own system, as well as why we chose to solve the problem this way…and whether or not we succeeded! — Maxfield F Stewart of Riot Games How to Do Continuous Delivery with Jenkins Pipeline, Docker and Kubernetes September 15th 2:30 PM - 3:15 PM, Great America J In this talk, we’ll show how to use Jenkins Pipeline together with Docker and Kubernetes to implement a complete end-to-end continuous delivery and continuous improvement system for microservices and monolithic applications using open source software. We’ll demonstrate how to easily create new microservices projects or import existing projects, have them automatically built, system and integration tested, staged and then deployed. Once deployed, we will also see how to manage and update applications using continuous delivery practices along with integrated ChatOps - all completely automated! — James Strachan of Red Hat Scaling Jenkins with Docker: Swarm, Kubernetes or Mesos? September 15th 2:30 PM - 3:15 PM, Exhibit Hall C The Jenkins platform can be dynamically scaled by using several Docker cluster and orchestration platforms, using containers to run agents and jobs and also isolating job execution. But which cluster technology should be used? Docker Swarm? Apache Mesos? Kubernetes? How do they compare? All of them can be used to dynamically run jobs inside containers. This talk will cover these main container clusters, outlining the pros and cons of each, the current state of the art of the technologies and Jenkins support. I believe people will be very interested in learning about the multiple options available. — Carlos Sanchez of CloudBees So, You Want to Build the World’s Biggest Jenkins Cluster? September 15th 3:45 PM - 4:30 PM, Exhibit Hall C How can we do it? We start with some real world results realized by Jenkins users who have built large clusters and review how they got there. Next, we will do experiments scaling some individual sub-components of Jenkins in isolation and see what challenges we will face when integrated. The famous large, distributed systems undoubtedly faced problems scaling - and we can learn from them, too. The result will be recipes for building Jenkins clusters with different scaling capabilities. After all of this, you can build the biggest Jenkins cluster in the world…or maybe just make your own Jenkins cluster more efficient. — Stephen Connolly of CloudBees Jenkins at Splunk and Splunking Jenkins September 15th 3:45 PM - 4:30 PM, Exhibit Hall A-1 This session will highlight how Splunk uses Jenkins to provide an end-to-end solution in the development CI system. Attendees will see how test results are delivered to a Splunk indexer, where they can be analyzed and presented in a variety of ways. This session will also include a live demonstration. — Bill Houston of Splunk Jenkins inside Google September 15th 4:45 PM - 5:30 PM, Exhibit Hall C Last year, we presented our initial investigations and stress testing as we prepared to deploy a large-scale Jenkins installation at Google. Now, with a year of real-world use under our belts, we’ll discuss how our expectations held up, what new issues we encountered and how we have addressed them. — David Hoover of Google In addition to these, we will also be hosting a Jenkins World Contributor Summit where "scaling" relevant topics such as "Storage Pluggability" will be discussed. The Jenkins World agenda is packed with even more sessions, so it should be a very informational event for everybody; hope to see you there!

At this year’s Jenkins World, our events officer Alyssa has been working to organize various activities in the "Open Source Hub" on the expo floor. Both days of the conference (Sept. 14th and 15th), during the break for lunch, there will be 15 minute demos by many of the experts helping to staff the Open Source Hub. Demo Schedule Wednesday, September 14th Time Session Details Presenter 12:15 - 12:30 Blue Ocean in Action Showcase of Blue Ocean and how it will make Jenkins a pleasure to use. Keith Zantow 12:30 - 12:45 Notifications with Jenkins Pipeline Sending information to Slack, HipChat, email and more from your Pipeline Liam Newman 12:45 - 13:00 Docker and Pipeline Learn how to use Docker inside of Pipeline for clean, repeatable testing environments R Tyler Croy 13:00 - 13:15 Git plugin - large repos, submodule authentication and more Techniques for managing large Git repositories, Submodule authentication, Pipelines and Git Mark Waite 13:15 - 13:30 Freestyle to Pipeline Overview of how easy it is to migrate from a confusing series of Freestyle Jobs to Jenkins Pipeline R Tyler Croy 13:30 - 13:45 package.json and Jenkins Using package.json to control your build; running tests, coverage and generating documentation in Jenkins Casey Vega 13:45 - 14:00 Extending Pipeline with Libraries When you have many jobs using similar configuration, it is natural to factor out the common parts into libraries. See some ways Pipeline lets you do this. Jesse Glick Thursday, September 15th Time Session Details Presenter 12:15 - 12:30 A simpler way to define Jenkins Pipelines Get to know a new way to define your Pipelines in a more configuration-like way! Andrew Bayer 12:30 - 12:45 Multibranch Pipelines + Git symbolic-ref Pipeline Multibranch Plugin is amazing, but is even better when used with Git symbolic references. The combination of the two gives users a way to create individual Jenkins jobs for each of their build/test configurations, instead of using a single parameterized job. I’ll show how to use these tools together to home in on problematic tests, systems under test, or both. Jon Hermansen 12:45 - 13:00 External Workspace Manager plugin for Jenkins Pipeline Meet the External Workspace Manager plugin, which supports managing workspaces across multiple Jenkins jobs running on different nodes and more! Alex Somai 13:00 - 13:15 Ownership plugin for Jenkins The presentation will introduce the Ownership engine for Jenkins jobs, folders and nodes. The presentation will cover plugin WebUI features, Ownership-based security and integration with Jenkins Pipeline Oleg Nenashev 13:15 - 13:30 Pipelines for building and deploying Android apps Using the various Android-related plugins for Jenkins, we will demonstrate pipelines to automatically build, test, and securely deploy Android apps. Christopher Orr As you can see there is a lot to see in the Open Source Hub at Jenkins World. To my knowledge these demos are not going to be recorded, so your only opportunities to see them might be at Jenkins World or your local Jenkins Area Meetup! Register for Jenkins World in September with the code JWFOSS for a 20% discount off your pass.

This is a guest post by Jenkins World speaker David Hinske, Release Engineer at Goodgame Studios. Hey there, my name is David Hinske and I work at Goodgame Studios (GGS), a game development company in Hamburg, Germany. As Release Engineer in a company with several development teams, it comes in handy using several Jenkins instances. While this approach works fine in our company and gives the developers a lot of freedom, we came across some long-term problems concerning maintenance and standards. These problems were mostly caused by misconfiguration or non-use of plugins. With “configuration as code” in mind, I took the approach to apply static code analysis with the help of SonarQube, a platform to manage code quality, for all of our Jenkins job configurations. As a small centralized team, we were looking for an easy way to control the health of our growing Jenkins infrastructure. With considering “configuration as code“, I developed a simple extension of SonarQube, to manage the quality and usage of all spawned Jenkins instances. The given SonarQube features (like customized rules/metrics, quality profiles and dashboards) allow us and the development teams to analyze and measure the quality of all created jobs in our company. Even though Jenkins configuration analysis cannot cover all SonarQube’s axes of code quality, I think there is still potential for conventions/standards, duplications, complexity, potential bugs (misconfiguration) and design and architecture. The results of this analysis can be used by all people working with Jenkins. To achieve this, I developed a simple extension of SonarQube, containing everything which is needed to hook up our SonarQube with our Jenkins environment. The implementation contains a new basic-language “Jenkins“ and an initial set of rules. Of course the needs depend strongly on the way Jenkins is being used, so not every rule implemented might be useful for every team, but this applies to all types of code analysis. The main inspirations for the rules were developer feedback and some articles found in the web. The different ways Jenkins can be configured provides the potential for many more rules. With this new approach of quality analysis, we can enforce best practices like: Polling must die (Better to triggerb uilds from pushes than poll the repository every x minutes). Use Log Rotator (Not using log-rotator can result in disk space problems on the controller). Use agents/labels (Jobs should be defined where to run). Don’t build on the controller (In larger systems, don’t build on the controller). Enforce plugin usage (For example: Timestamp, Mask-Passwords). Naming sanity (Limit project names to a sane (e.g. alphanumeric) character set). Analyze Groovy Scripts (For example: Prevent System.exit(0) in System Groovy Scripts). Besides taking control of all configuration of any Jenkins instance we want, there is also room for additional metrics, like measuring the amount and different types of jobs (Freestyle/Maven etc…​) to get an overview about the general load of the Jenkins instance. A more sophisticated idea is to measure complexity of jobs and even pipelines. As code, jobs configuration gets harder to understand the more steps are involved. On the one hand scripts, conditions and many parameters can negatively influence the readability, especially if you have external dependencies (like scripts) in different locations. On the other hand, pipelines can also grow very complex when many jobs are involved and chained for execution. It will be very interesting for us to see where and why too complex pipelines are being created. On visualization we rely on the data and its interpretation of SonarQube, which offers a big bandwidth of widgets. Everybody can use and customize the dashboards. Our centralized team for example has a separate dashboard where we can get a quick overview over all instances. The problem of "growing" Jenkins with maintenance problems is not new. Especially when you have many developers involved, including with the access to create jobs and pipelines themselves, an analysis like this SonarQube plugin provides can be useful for anyone who wants to keep their Jenkins in shape. Customization and standards are playing a big role in this scenario. This blog post surely is not an advertisement for my developed plugin, it is more about the crazy idea of using static code analysis for Jenkins job configuration. I haven’t seen anything like it so far and I feel that there might be some potential behind this idea. Join me at my Enforcing Jenkins Best Practices session at the 2016 Jenkins World to hear more! David will be presenting more of this concept at Jenkins World in September. Register with the code JWFOSS for 20% off your full conference pass.

Our events officer Alyssa has been working for the past several weeks to organize the "Open Source Hub" at Jenkins World 2016. The Hub is a location on the expo floor where contributors to the Jenkins project can hang out, share demos and help Jenkins users via the "Ask the Experts" program. Thus far we have a great list of experts who have volunteered to help staff the booth, which includes many frequent contributors, JAM organizers and board members. A few of the friendly folks you will see at Jenkins World are: Paul Allen - P4 Plugin maintainer and Pipeline contributor. R Tyler Croy - Jenkins infrastructure maintainer and board member. Jesse Glick - Pipeline maintainer and long-time contributor to Jenkins core. Eddú Meléndez Gonzales - Organizer for the Lima (Perú) Jenkins Area Meetup and contributor to Spring. Jon Hermansen - Organizer for the Los Angeles Jenkins Area Meetup, developer and Pipeline user. Owen Mehegan - GitLab plugin contributor, release engineer and copy editor for jenkins.io. Oleg Nenashev - Google Summer of Code organizer, maintainer of multiple plugins and St. Petersburg Jenkins Area Meetup organizer. Christopher Orr - Maintainer of multiple Android-related plugins, including the Android Emulator plugin and contributor to numerous projects behind the scenes of Jenkins. Casey Vega - Organizer for the Los Angeles Jenkins Area Meetup and release engineer at Verizon Digital Media. Mark Waite - Maintainer of the Git plugin and contributor to a number of other Git-related plugins. Dean Yu - Long-time contributor, board member and release engineer at Shutterfly. I hope that this list isn’t exhaustive! If you are an active member of the Jenkins community and/or a contributor, consider taking part in the "Ask the Experts" program. It’s a great opportunity to bond with other contributors and talk with fellow users at Jenkins World. You will be able to find us in the expo hall under the "Open Source Hub" sign; please stop by at Jenkins World to say hello, pick up stickers and to ask questions! Register for Jenkins World in September with the code JWFOSS for a 20% discount off your pass.

At Jenkins World 2016 on September 14-15, stop by the "Open Source Hub", located in the Partner Expo hall at the Santa Clara Convention Center in Santa Clara, CA. The Open Source Hub will have many Jenkins contributors, committers, JAM leaders, and officers from the governance board under one roof, so there will be plenty of knowledge and talents on hand to share. We hope you’ll join in on the festivities. Ask the Experts The setup that is waiting for you: white boards, monitors and lots of brain power to help answer those Jenkins questions that have been keeping you up at night. Jenkins experts can help with beginner questions to the more advanced ones. All you need to do is bring your laptop and your questions; the experts will help answer them! Register for Jenkins World in September with the code JWFOSS for a 20% discount off your pass. Live Demos Sometimes seeing is believing, there will be plenty of demos in the "Open Source Hub" during the lunch hours on Wednesday September 14th, and Thursday September 15th in the expo hall. Jenkins experts will be show-casing their favorite Jenkins features, plugins and projects. Grab your lunch, take a seat in the open source theater to learn about: Pipelines for Building and Deploying Android Apps by Android Emulator plugin maintainer Chris Orr Git Plugin - Large Repos, Submodule Authentication, and more by Git plugin maintainer Mark Waite Docker and Pipeline by Jenkins infrastructure contributor R Tyler Croy Extending Pipeline with Libraries by Pipeline plugin maintainer Jesse Glick Blue Ocean in Action by Blue Ocean contributor Keith Zantow External Workspace Manager plugin for Pipeline by Google Summer of Code student Alexandru Somai And many more Jenkins Mural Jenkins World participants will take part in the realization of a giant collaborative mural painting with the CommitStrip team. Thomas, the writer and Etienne, the cartoonist, teamed up with a few Jenkins contributors to design a 5m x 2m mmural which will be drawn live! Brushes and colors will be available for all attendees who wish to help paint this one of a kind piece of Jenkins art. Sticker Swap Jenkins World attendees will have a chance to swap stickers. There will be a table where attendees are welcome to place/take stickers. Bring your cool stickers to share with others and take stickers that interest you. After Dark Reception Sponsored by CloudBees After Dark reception will be from 6-8pm on Wed Sept 14 in the Partner Expo. Enjoy cocktails, appetizers, mingle, and dance to a live band. A big THANK YOU goes out to CloudBees for their generous contributions! See you at After Dark! Contributor Summit - Tuesday, September 13 If Blue Ocean, Pipeline and Storage Pluggability sounds interesting to you, join the interactive discussions surrounding these topics. The Jenkins project is also looking to hear use-cases, war stories, and pain points. The objective of the summit is to work towards improving the Jeknins project. Seats are limited. Don’t forget to register; I look forward to seeing you at the conference! Links Jenkins World 2016 Acknowledgements Special thanks to CloudBees as the premier sponsor and BlazeMeter, Microsoft, Red Hat and all the other sponsors who have made this event possible.

This is a guest post by Jenkins World speaker Neil Hunt, Senior DevOps Architect at Aquilent. In smaller companies with a handful of apps and fewer silos, implementing CD pipelines to support these apps is fairly straightforward using one of the many delivery orchestration tools available today. There is likely a constrained tool set to support - not an abundance of flavors of applications and security practices - and generally fewer cooks in the kitchen. But in a larger organization, I have found that in the past, there were seemingly endless unique requirements and mountains to climb to reach this level of automation on each new project. Neil will be presenting more of this concept at Jenkins World in September, register with the code JWFOSS for a 20% discount off your pass. Enter the Jenkins Pipeline plugin. My recently departed former company, a large financial services organization with a 600+ person IT organization and 150+ application portfolio, set out to implement continuous delivery enterprise-wide. After considering several pipeline orchestration tools, we determined the Pipeline plugin (at the time called Workflow) to be the superior solution for our company. Pipeline has continued Jenkins' legacy of presenting an extensible platform with just the right set of features to allow organizations to scale its capabilities as they see fit, and do so rapidly. As early adopters of Pipeline with a protracted set of requirements, we used it both to accelerate the pace of onboarding new projects and to reduce the ongoing feature delivery time of our applications. In my presentation at Jenkins World, I will demonstrate the methods we used to enable this. A few examples: We leveraged the Pipeline Remote File Loader plugin to write shared common code and sought and received community enhancements to these functions. Jenkinsfile, loading a shared AWS utilities function library awsUtils.groovy, snippets of some AWS functions We migrated from EC2 agents to Docker-based agents running on Amazon’s Elastic Container Service, allowing us to spin up new executors in seconds and for teams to own their own executor definitions. Pipeline run #1 using standard EC2 executors, spinning up EC2 instance for each node; Pipeline run #2 using shared ECS cluster with near-instant instantiation of a Docker agent in the cluster for each node. We also created a Pipeline Library of common pipelines, enabling projects that fit certain models to use ready-made end-to-end pipelines. Some examples: Maven JAR Pipeline: Pipeline that clones git repository, builds JAR file from pom.xml, deploys to Artifactory, and runs maven release plugin to increment next version Anuglar.JS Pipeline: Pipeline that executes a grunt and bower build, then runs S3 sync to Amazon S3 bucket in Dev, then Stage, then Prod buckets. Pentaho Reports Pipeline: Pipeline that clones git repository, constructs zip file, and executes Pentaho Business Intelligence Platform CLI to import new set of reports in Dev, Stage, then Prod servers. Perhaps most critically, a shout-out to the saving grace of this quest for our security and ops teams: the manual 'input' step! While the ambition of continuous delivery is to have as few of these as possible, this was the single-most pivotal feature in convincing others of Pipeline’s viability, since now any step of the delivery process could be gate-checked by an LDAP-enabled permission group. Were it not for the availability of this step, we may still be living in the world of: "This seems like a great tool for development, but we will have a segregated process for production deployments." Instead, we had a pipeline full of many 'input' steps at first, and then used the data we collected around the longest delays to bring management focus to them and unite everyone around the goal of strategically removing them, one by one. Going forward, having recently joined Aquilent’s Cloud Solutions Architecture team, I’ll be working with our project teams here to further mature the use of these Pipeline plugin features as we move towards continuous delivery. Already, we have migrated several components of our healthcare.gov project to Pipeline. The team has been able to consolidate several Jenkins jobs into a single, visible delivery pipeline, to maintain the lifecycle of the pipeline with our application code base in our SCM, and to more easily integrate with our external tools. Due to functional shortcomings in the early adoption stages of the Pipeline plugin and the ever-present political challenges of shifting organizational policy, this has been and continues to be far from a bruise-free journey. But we plodded through many of these issues to bring this to fruition and ultimately reduced the number of manual steps in some pipelines from 12 down to 1 and brought our 20+ Jenkins-minute pipelines to only six minutes after months of iteration. I hope you’ll join this session at Jenkins World and learn about our challenges and successes in achieving the promise of continuous delivery at enterprise scale.