The cloud-native era has opened up a whole new level for CI/CD, and it’s impacting the design of our pipelines. The days of the big monolith pipeline that only one person can decipher are gone. Let’s see how we can design and implement cloud-native CI/CD pipelines on Kubernetes, for the enterprise, using Jenkins X.

Requirements

Let’s review our requirements first. I said “for the enterprise”, because writing pipelines for a single open-source project or hundreds/thousands of enterprise projects is not the same.

The first requirement is the conventions or standards you might have in your company around code quality, testing, packaging…

One year ago, we wrote about our journey from Jenkins to Jenkins X. It’s time to take a step back and see where we are now and how this journey has impacted the way we write and deliver software at Dailymotion.

We’ve been using Jenkins X for more than one year now to handle all the build and delivery of our ad-tech platform at Dailymotion — with great results. Applying the practices described in the Accelerate book and implemented in Jenkins X allowed us to break the silos, to “shift left” some responsibilities and move faster. …

Our use-case

We’re using both Kubernetes to deploy our applications and Okta as a company SSO. We’re also big fans of Jenkins X. Jenkins X comes with a few UI, which unfortunately don’t have native authentication/authorization implementation yet:

• CloudBees Jenkins X UI
• Deck — Prow’s dashboard
• and we also deployed Tekton Dashboard

Jenkins X relies on Nginx for its ingress controller, and it uses the basic auth feature to protect its UI by default. The issue with this solution is that you either need to manually manage all your users (and passwords), or give them a shared set of credentials.

As we’re…

In the previous part of this series, we’ve walked through the internals of the stages that compose a pipeline. Now we’ll see how the Jenkins X Steps — that compose a stage — are implemented.

First, a few pointers:

• the reference documentation for the steps
• the source code definition of a step

These steps — we’ll call them Jenkins X Steps — are converted by the meta pipeline into Tekton Steps. And a Tekton Step is just a Kubernetes Container, as you can see in the source code definition of a Tekton Step.

So:

• Jenkins X Stages are converted into…

In the first part of this series, we’ve walked through everything that happened in the cluster, from the incoming GitHub WebHook event to a running Tekton pipeline. In the second part, we’ve talked about the “Meta Pipeline”, which is responsible for converting your Jenkins X Pipeline into a Tekton Pipeline. Now it’s time to dive into the internals of the Tekton pipelines, and we’ll start with an investigation of how the Jenkins X Stages — that compose a pipeline — are implemented.

So a pipeline has stages, and each stage has steps. But why do we need stages? Well, for…

In the first part of this series, we’ve walked through everything that happened in the cluster, from the incoming GitHub WebHook event to a running Tekton pipeline. But pipelines can be complex, and built from multiple levels of inheritance. This is the case with the Jenkins X Pipelines, which supports Build Packs to abstract away most of the complexity for the developers. The downside is that building the whole pipeline is now more complex because it requires retrieving parts of it from multiple git repositories — before combining them all together. Doing all that work is too much for the…

As I am starting to migrate from the Jenkins Declarative Pipelines to the (new) Jenkins X Pipelines — defined in YAML format and implemented using Prow and Tekton — I am diving into the internals of the system, because I like to understand how things works.

So I am now sharing my findings here, as a series of blog posts. I’ll add links to source code so you can dive deeper into a specific topic if you want.

• The first part is a walk-through of a GitHub WebHook event coming to the cluster until it results in a running Tekton…

The “new” Jenkins X Pipelines — starting from Jenkins X 2.0 — are implemented using open-source components such as Prow and Tekton. In a few words, Prow has been built into and for Kubernetes, and is responsible for all the GitHub-related automation: reacting to GitHub events, triggering Jobs to run tests, automatically merging Pull Requests, Chat-Ops style user-interface, and so on. It has been battle-tested by the Kubernetes community, which is one of the biggest open-source communities out there. And Tekton is a young project extracted from the Knative project, designed to provide the building blocks for running Pipelines on…