Opinionated Kubernetes and Jenkins X

    I recently wrote about how all the cloud platforms are all in Kubernetes and so are developers. It is an exciting time, but the problem for many is that this is a huge blank sheet of paper for how to build and deploy applications. A white space, a void, a limitless canvas of possibilities. Insert metaphors here.

    The problem, as you may guess, is that few people really like or are able to start with a blank canvas. I know I prefer to start with something working and iterate towards a solution, or be given some rails to stay on (again with the metaphors).

    That’s where the Jenkins X project comes in. Jenkins X is a Kubernetes-native continuous integration and continuous delivery platform for developing cloud native applications that was recently introduced as a Jenkins Enhancement Proposal, sponsored by James Strachan.

    There is a lot to take in but at it’s heart, this is an open source opinionated way to do continuous delivery with Kubernetes, natively, without necessarily having to learn all the things I talked about in my last blog post. I shall attempt to explain what this is all about and why it matters to developers. As someone said on the jenkins-dev mailing list “We have the two glued together with baling wire and twine” - Jenkins X aims to simplify how to work with continuous delivery and Kubernetes.

    First and most importantly, let’s see the logo:

    Jenkins X Logo

    You can see the nautical theme leaking through (and Kubernetes). Whilst it is called Jenkins X, it is about quite a lot more than Jenkins.

    Jenkins X makes decisions for you

    Jenkins X presents itself to you initially as a handy sleek command line (a native binary you can install called jx - the debate is on as to how pronounce it). Let’s take a tour (sail?):

    > jx import my-app

    If you have an existing project, this will detect what type of project it is, build a pipeline for you (and a bunch of Kubernetes things, like Helm Charts), add it to your project and set it up in GitHub, WebHooks and all, build the project (run the pipeline) and deploy a version to a “staging” environment.

    If it looks ok, you can promote it to production:

    > jx promote --env production --version 1.0.1 my-app

    If something went wrong in production, you can roll back an app to any version (the version numbers are made for you):

    > jx promote --env production --version 1.0.0 my-app
    > jx get apps # list versions

    An environment is a well-established concept for web developers using continuous delivery: out of the box Jenkins X makes three of them for you (dev, staging and production), but you can make as many as you like. Environments have rules around how things are promoted into them (and they also have their own extensible pipelines, but you can just use them as-is to start).

    You can also create a Spring Boot microservice app:

    > jx create spring

    Answer a few questions and it will set everything up for you.

    Any changes you make to your app automatically are built, and if the build looks good, they go through to the staging environment. WebHooks are setup for you (if you are using GitHub) to smooth this over.

    For those looking at starting from pre-made apps, there are "quickstarts":

    > jx create quickstart

    They are based on a growing set of starter apps, in a variety of languages and tech stacks.

    Review apps for your changes: Each pull request is built/tested, and a “review app” is made available in a temporary environment. That means each proposed change, before it goes to the default branch (master), has an environment made (temporary) that it can be tried out in. In GitHub, this shows up as a comment in the pull request:

    Comment in a GitHub Pull Request

    Project type detection

    As you can see, so far there is no editing or manual creation of pipelines, or scripting or setup, just importing or creating your app and go. This is powered by Draft “packs” (a handy project that came out of Azure). What you end up with is a Jenkinsfile in your project repository. You may want to edit it one day, or you may be happy with it as is! Jenkins is famous for being un-opinionated in what you do, but Jenkins X has strong opinions (but you can extend and customise).

    image::/images/jenkins-x/draft-logo.png[Draft Logo, width=300]

    Deploying or promoting to environments

    Deploying happens via pipelines behind the scenes - when a change is pushed, or a version promoted. You don’t need to directly interact with Kubernetes if you don’t need to. A tool called Helm does the heavy lifting: Helm is used to package and perform installations and upgrade for your apps.

    Helm Logo

    There is a little more magic going on here with environments, which you don’t see at first. Each environment, for a team, is represented by a Git repository behind the scenes. Configuration as code is a well-established best practice these days, so why not use it to track deployments and initiate deployments. I also mentioned in my previous post how declarative Kubernetes is: it is perfect for keeping all config in a repository, of the desired system state.

    Each promotion is actually a pull request to a per-environment repository. This repository is made and managed for you (and kept outside of the main application code repository), you don’t have to look at it, but you can extend things there should you need to. Specific environment repositories may have different access rules, or be controlled by a different team (perhaps even deploy to a different cluster). Some have coined the term for this as “GitOps.” I first came across this concept on a WeaveWorks blog.

    I’ll try and explain this one with a diagram:

    GitOps Workflow

    The pipeline is actually split in the middle. On the left is the more familiar continuous integration pipeline. This works on pull requests, pre-release version of things and is all about testing(automated and manual review). The source of truth for this is the configuration in the applications repository: branches, pull requests and so on.

    The right-hand side is the continuous delivery pipeline. This kicks in when the application is ready to be updated with a new release. This is the “GitOps” repo behind the scenes that controls the state of things in Kubernetes. A promotion on this side is a pull request, and then a merge, from the staging repository to the production repository.

    Installing Jenkins X

    The jx command line has a jx install command that installs it into a Kubernetes cluster.

    The best experience initially is using Google’s excellent GKE service:

    > jx create cluster gke

    This will ask a few questions, and go and set it all up for you in a cluster set aside for Jenkins X (recommended). Jenkins X runs entirely as services on top of a Kubernetes cluster.

    > jx install

    Is designed to work with a Kubernetes cluster (if it already exists, recommendation is to have a cluster put aside for Jenkins X if possible). Amazon EKS support is coming (mostly it is around testing), that service is in beta/early access so it is still a work in progress, as is Microsoft Azures excellent AKS service.

    So where is Jenkins?

    Good question, thanks for asking. Well, it is behind the scenes. As you have seen, there was no direct interaction with Jenkins, but it is there, running the pipelines for continuous integration and continuous delivery of the respective repositories, and orchestrating things with Kubernetes.

    If you run jx get pipelines you can see URLs to the various pipelines that have been setup for you are part of interacting with Jenkins X.

    By the way, James Strachan has written an extensive blog on jenkins.io that really explores the Jenkins X project in-depth. Once you finish reading this blog, take a stroll on over there and read James'. He also provides several ways you can get involved in the project.

    What else can I do with the command line?

    Lots, the jx command line has built in help:

    jx open

    • open apps, services or pipelines in your browser

    jx activity

    • explains how things got to where they are, a history

    jx get environments

    • list environments

    jx get apps

    • show the state of applications, what versions are in what environments.

    What’s next

    There is a whole lot more to this, and lots more moving parts and services that are set up for you that are very useful, but it is best to head over to jenkins-x.io and have a look.

    This project is obviously in early stages (it is stll a Draft JEP after all) and there is lots happening. Check out the Jenkins X community if you want to chat on slack, IRC, issues or email. Also, read the Jenkins Enhancement Proposal doc.

    About the Author
    Michael Neale
    Michael Neale

    Michael is a CD enthusiast with a interest in User Experience. He is a co-founder of CloudBees and a long time OSS developer, and can often be found lurking around the jenkins-dev mailing list or #jenkins on irc (same nick as twitter name). Before CloudBees he worked at Red Hat.