Jenkins X uses Capabilities identified by the "Accelerate: The Science Behind Devops" Jenkins X is a reimagined CI/CD implementation for the Cloud which is heavily influence by the State of DevOps reports and more recently the book "Accelerate: The Science Behind Devops" by Nicole Forsgren, Jez Humble and Gene Kim Years of gathering data from real world teams and organisations which has been analyzed by inspiring thought leaders and data scientists from the DevOps world, "Accelerate" recommends a number of capabilities that Jenkins X is implementing so users gain the scientifically proven benefits, out of the box. We’ve started documenting the capabilities that are available today and will continue as more become available. Credit: thanks to tracymiranda for the image Use version control for all artifacts The Weaveworks folks coined the term GitOps which we love. Any change to an environment, whether it be a new application, version upgrade, resource limit change or simple application configuration should be raised as a Pull Request to Git, have checks run against it like a form of CI for environments and approved by a team that has control over what goes into the related environment. We can now enable governance and have full traceability for any change to an environment. Related Accelerate capability: Use version control for all production artifacts Automate your deployment process Environments Jenkins X will automatically create Git backed environments during installation and makes it easy to add new ones using jx create environment. Additionally when creating new applications via a quickstart ( jx create quickstart), Java based SpringBoot ( jx create spring) or importing existing applications ( jx import), Jenkins X will both automatically add CI / CD pipelines and setup the jobs, git repos and webhooks to enable an automated deployment process. Out of the box Jenkins X creates Staging and Production (this is customisable) permanent environments as well as temporary environments for preview applications from Pull Requests. Previews Environments We are trying to move as much testing, security, validation and experimentation for a change before it’s merged to master. With the use of temporary dynamically created Preview Environments any pull request can have a preview version built and deployed, including libraries that feed into a downstream deployable application. This means we can code review, test, collaborate better with all teams that are involved in agreeing that change can go live. Ultimately Jenkins X wants to provide a way that developers, testers, designers and product managers can be as sure as they can that when a change is merged to master it works as expected. We want to be confident the proposed change does not negatively affect any service or feature as well as deliver the value it is intended to. Where Preview Environments get really interesting is when we are able to progress a PR through various stages of maturity and confidence where we begin to direct a percentage of real production traffic like beta users to it. We can then analyse the value of the proposed change and possible run multiple automated experiments over time using Hypothesis Driven Development. This helps give us better understanding of how the change will perform when released to all users. Related Accelerate capability: Foster and enable team experimentation Using preview environments is a great way to introduce better test automation. While Jenkins X enables this we don’t yet have examples of automated tests being run against a preview environment. A simple test would be to ensure the application starts ok and Kubernetes liveness check pass for an amount of time. This relates to Related Accelerate capability: Implement Test Automation Related Accelerate capability: Automate your deployment process Permanent Environments In software development we’re used to working with multiple environments in the lead up to a change being promoted to a live production environment. Whilst this seems business as usual it can cause significant delays to other changes if for any reason that it is deemed not fit via some process that didn’t happen pre merge to master. Subsequent commits then become blocked and can cause delay of urgent changes being promoted to production. As above Jenkins X wants any changes and experiments to be validated before it is merged to master. We would like changes in a staging environment to be held there for a short amount of time before being promoted, ideally in an automated fashion. The default Jenkins X pipelines provide deployment automation via environments. These are customisable to suite your own CI / CD pipeline requirements. Jenkins X recommends Staging should act as a near as possible reflection on production, ideally with real production data shadowed to it using a service mesh to understand the behaviour. This also helps when developing changes in preview where we can link to non production services in staging. Related Accelerate capability: Automate your deployment process Use trunk-based development The Accelerate book found that teams which use trunk based development with short lived branches performed better. This has always worked for the Jenkins X core team members so this was an easy capability for Jenkins X to implement when setting up Git repositories and CI/CD jobs. Implement Continuous Integration Jenkins X sees CI as the effort of validating a proposed change via pull requests before it is merged to controller. Jenkins X will automatically configure source code repositories, Jenkins and Kubernetes to provide Continuous Integration of the box. Implement Continuous Delivery Jenkins X sees CD as the effort of taking that change after it’s been merged to controller through to running in a live environment. Jenkins X automates many parts in a release pipeline: Jenkins X advocates the use of semantic versioning. We use git tags to calculate the next release version which means we don’t need to store the latest release version in the controller branch. Where release systems do store the last or next version in Git repos it means CD becomes hard, as a commit in a release pipeline back to controller triggers a new release. This results in a recursive release trigger. Using a Git tag helps avoid this situation which Jenkins X completely automates. Jenkins X will automatically create a released version on every merge to master which can then potentially progress through to production. Use loosely coupled architecture By targeting Kubernetes users of Jenkins X can take advantage of many of the cloud features that help design and develop loosely coupled solutions. Service discovery, fault tolerance, scalability, health checks, rolling upgrades, container scheduling and orchestration to name just a few examples of where Kubernetes helps. Architect for empowered teams Jenkins X aims to help polyglot application developers. Right now Jenkins X has quickstarts and automated CI/CD setup with language detection for Golang, Java, NodeJS, .Net, React, Angular, Rust, Swift and more to come. What this also does is provide a consistent Way of Working so developers can concentrate on developing. Jenkins X also provides many addons, for example Grafana and Prometheus for automated metrics collection and visualisation. In this example centralised metrics help understand how your applications behave when built and deployed on Kubernetes. DevPods are another feature which enables developers to edit source code in their local IDE, behind the scenes it is then synced to the cloud and rapidly built and redeployed. Jenkins X believes providing developers automation that helps them experiment in the cloud, with different technologies and feedback empowers them to make the best decisions - faster. Fancy a closer look? Myself, James Strachan and Rob Davies are going to be presenting and running workshops at DevOps World | Jenkins World. We’ll also be hanging out at the Jenkins X demo area so come and say hello and see what’s the latest cool and exiting things to come out of Jenkins X. Use JWFOSS for 30% discount off registration Want to get involved? Jenkins X is open source, the community mainly hangs out in the Jenkins X Kubernetes slack channels and for tips on being more involved with Jenkins X take a look at our contributing docs. We’ve been helping lots of folks get into open source, learn new technoligies and languages like golang. Why not get involved? Demo If you’ve not already seen it here’s a video showing a spring boot quickstart with automatic CI/CD pipelines and preview environments.

Anchore provides docker image analysis for user defined acceptance policies to allow automated image validation and acceptance. As developers we would like to know if a change we are proposing introduces a Common Vulnerability and Exposure (CVE). As operators we would like to know what running applications are affected if a new CVE is discovered. Now in Jenkins X pipelines, if we find an Anchore engine service running we will add the preview and release images to be analyzed. This means we can look at any environment including previews (created from Pull Requests) to see if your application contains a CVE. Upgrade Start by checking your current Jenkins X version: jx version If your Jenkins X platform is older than 0.0.903, then first you will need to upgrade to at least 0.0.922: jx upgrade cli jx upgrade platform Install addon You can install the Anchore engine addon when you are in your Jenkins X team home environment. jx env dev jx create addon anchore This will install the engine in a seperate anchore namespace and create a service link in the current team home environment so our pipeline builds can add docker images to Anchore for analysis. Create an application You can now create a new quickstart: jx create quickstart List any CVEs Once the build has run you will be able to check for CVEs in any environment incluing previews created for pull requests. jx get cve --environment staging Demo Here’s a 4 minute video that demonstrates the steps above: Upgrading existing pipelines If you have an existing application pipeline and and want enable image analysis you can update your Jenkinsfile, in the preview stage after the skaffold step add the line sh "jx step validate --min-jx-version 1.2.36" sh "jx step post build --image \$JENKINS_X_DOCKER_REGISTRY_SERVICE_HOST:\$JENKINS_X_DOCKER_REGISTRY_SERVICE_PORT/$ORG/$APP_NAME:$PREVIEW_VERSION" In the master stage the add this line after the skaffold step sh "jx step validate --min-jx-version 1.2.36" sh "jx step post build --image \$JENKINS_X_DOCKER_REGISTRY_SERVICE_HOST:\$JENKINS_X_DOCKER_REGISTRY_SERVICE_PORT/$ORG/$APP_NAME:\$(cat VERSION)" For any questions please find us - we mainly hang out on Slack at #jenkins-x-dev - or see jenkins-x.io/community for other channels.

Jenkins X is an opinionated platform for providing CI / CD on top of Kubernetes. We’ve chosen a set of core applications that we install and wire together so things work out-of-the-box, providing a turn key experience. This blog aims to build on previous introductions to Jenkins X and provide a deeper insight to what you get when you install Jenkins X. So what happens? After downloading the jx CLI you will now be able to create clusters with public cloud providers or install onto an existing Kubernetes cluster. This command will create a cluster on your cloud provider of choice. jx create cluster Alternatively you can bring your own Kubernetes cluster and install Jenkins X on it: jx install That said, we’ve found that creating a new cluster on a public cloud such as GKE is a lot way easier to start as we can be sure of the state of the cluster. For example we know that storage, networking and loadbalancers will be working as expected. Creating a cluster on GKE takes only a few minutes so it’s a great way to try things out as well as run your enterprise workloads. For now lets assume we are using GKE. When jx create cluster has finished you will see some output in the terminal that also includes the default admin password to use when logging into the core applications below. There is a flag --default-admin-password you can use to set this password yourself. Accessing applications We automatically install an Nginx ingress controller running with an external loadbalancer pointing at it’s Kubernetes service. We also generate all the Kubernetes Ingress rules using a golang library called " exposecontroller". This runs as a Kubernetes Job triggered by a Helm hook once any application is installed to the cluster. Using "exposecontroller" means we can control all the ingress rules for an environment using a single set of configurations, rather than each application needing to know how to expose the kubernetes service to the outside world. This also means we can easily switch between HTTP and HTTPS plus support intregration with projects like cert-manager for auto generation of signed TLS certificates. Environments One important point to make is Jenkins X aims to use terminology that developers are familiar with. That’s not to say we are changing Kubernetes fundamentals, it’s more that if you don’t know Kubernetes concepts then we aim to help you still adopt the cloud technology and pull back the curtain as you gain confidence and experience. To that point, a core part of Jenkins X are "environments". An environment can have one or more applications running in it. In Kubernetes term an "environment" maps to the concept of a "namespace" in code. The installation by default created three environments, this is customisable but by default we have a "dev", a "staging" and a "production environment". To list, select, or switch between these environments run: jx env Jenkins X core applications In the "dev" environment we have installed a number of core applications we believe are required at a minimum to start folks off with CI/CD on Kubernetes. We can easily add to these core apps using Jenkins X addons but for now lets focus on the core apps. Jenkins X comes with configuration that wires these services together, meaning everything works together straight away. This dramatically reduces the time to get started with Kubernetes as all the passwords, environment variables and config files are all setup up to work with each other. Jenkins — provides both CI and CD automation. There is an effort to decompose Jenkins over time to become more cloud native and make use of Kubernetes concepts around CRDs, storage and scaling for example. Nexus — acts as a dependency cache for Nodejs and Java applications to dramatically improve build times. After an initial build of a SpringBoot application the build time is reduced from 12 mins to 4. We have not yet but intend to demonstrate swapping this with Artifactory soon. Docker Registry — an in cluster docker registry where our pipelines push application images, we will soon switch to using native cloud provider registries such as Google Container Registry, Azure Container Registry or Amazon Elastic Container Registry (ECR) for example. Chartmuseum — a registry for publishing Helm charts Monocular — a UI used for discovering and running Helm charts Helm We learned a lot in our early days with fabric8 on Kubernetes and there were some projects from the ecosystem that either weren’t around or (at the time) didn’t work with OpenShift, therefore we were restricted when making some design decisions. A couple of years on and now with Jenkins X we were able to look at other OSS projects that have been flourishing, so I was very happy to start looking at Helm. Helm is a package manager for Kubernetes and allows easy installation and upgrades of applications. It was pretty clear that for Jenkins to evolve and include deployments to the cloud we should embrace Helm and provide an opinionated experience that helps teams and developers. The core applications mentioned above means Jenkins X provides an out of the box integrated CI/CD solution for Helm. We know that helm has limitations but with the work on Helm 3, the focus of the Kubernetes sig-apps group, the Kubernetes community and investment we see from key organisations such as Microsoft, we feel Helm is currently the best way to install and upgrade applications on Kubernetes. GitOps We mentioned earlier that we setup three environments by default. What this means is for the staging and production environments we created: Kubernetes namespace An environment resource ( CustomResourceDefinition) in the dev environment which includes details of how applications are promoted to it and includes various team settings. A git repository that we store what applications and their versions should be present in that environment. These are stored in a Helm requirements.yaml file A Jenkins Pipeline job: explained in more detail below CI/CD for Environments Having a Jenkins Pipeline Job for each environment means that Pull Requests to the git repo trigger a CI job. For now that job performs basic validation but in the future will include ‘gates’ to ensure a change to that environment has passed expected checks such as QA tasks, gain enough approvals from the correct people, etc - YES CI for environments! Once CI checks have passed the new application or version change can be merged. Only users that have karma can merge the Pull Request and therefore we get RBAC plus traceability for our environment deployments. This means every application manifest, their version and configuration including storage requirements, resource needs and secrets for your environments are stored in Git repositories. Given a disaster recovery scenario this is exactly what you want. Did I just say secrets in Git? Yes! We will be providing a nicer experience to helps folks get set up but we ourselves encrypt our secrets and store them in Git, then decrypt them when we come to install and upgrade. Here’s our Git repo https://github.com/jenkins-x/cloud-environments/blob/a1edcc6/env-jx-infra/secrets.yaml. We do all this with the help of a Helm wrapper called helm secrets. I’m working on a followup blog post with examples, better explanations and how to guides + add better integration with JX in the coming weeks. Fancy getting involved? We mainly hangout in the jenkins-x Kubernetes slack channels and for tips on being more involved with Jenkins X take a look at our contributing docs If you’ve not already seen it here’s a video showing the create cluster explained in this blog.