CI tools testing lab: Making it do useful work

screenshot-from-2016-12-06-17-32-07I this post I’m going to describe how to create Zuul-compatible Jenkins jobs and Zuul pipelines to trigger them on code changes in Gerrit.

This is the sixth and final post in a series describing how I’ve used Docker to build a lab setup on my laptop to try out Zuul and check out its various features. Previously I’ve described how to install Jenkins, Gerrit, add the zuul-server and zuul-merger components, link Jenkins to Zuul and show the Zuul graphical status page.

Before we can create Jenkins jobs and Zuul pipelines, we need to create a Gerrit project for everything to work against.

Creating a Gerrit project

There is not much to it really, as we’ve previously configured most of the needed settings for all projects. We just need to go to Gerrit`s web UI (At this lab system it would be at, sign in with an administrative account, go to “Projects” -> “Create New Project”, enter a project name and set it to inherit from “All-Projects”.

In my tests I’ve created the “zuul-test” project.

Gerrit project screenshot

Creating Zuul-compatible Jenkins jobs

There are many ways to create Jenkins jobs. For testing purposes I simply created them from the Jenkins web UI (Available at

A Zuul-compatible Jenkins job is triggered by Zuul itself, so no trigger setup is needed.

To gain the increased CI speed Zuul can provide we need to enable it to run jobs in parallel. So we need to check the “Execute concurrent builds if necessary” option when configuring the job.

Zuul passes some useful parameters to that job so we need to configure them in each job we want Zuul to invoke. The full list of parameters can be found in the Zuul documentation.

Zuul includes a “zuul-cloner” tool that can clone project source code as Zuul prepares it. But for single-project jobs we can also use Jenkins` Git plugin. To do that we configure its parameters as following:

  • Refspec (This will be hidden behind an “advanced” button): +$ZUUL_REF:refs/heads/zuul
  • Branch Specifier: refs/heads/zuul

I typically also prefer to clone to a directory rather then directly to the $WORKSPACE, so I add the “Check out to a sub-directory” custom behaviour and set “Local subdirectory for repo” to “$ZUUL_PROJECT”.

Setting up Jenkins Git plugin for Zuul

As far as the job’s build steps go, this is a test system, so there is not real build work to be done. I simply add an “Execute shell” build step to output some diagnostic information, stall for a while (So I can send multiple patches in parallel), and allow for a way to craft patches that will fail (So that failure behaviour can be tested).

env | egrep '(^ZUUL_|LOG_PATH$)'

for txt in $(find zool-test -name "*.txt"); do
        set +x
        echo ----------------------------------------------------------------
        echo "$txt"
        echo ----------------------------------------------------------------
        cat "$txt"

sleep 5m

! grep -Riq gate_fail "$ZUUL_PROJECT"/*

In total I’ve created two jobs – “zuul-check-patch” to run when a patchset is submitted to Gerrit, and “zuul-check-merged” to run when we indicate to zuul its time to gate-check the patch and merge it if successful.

Configuring Zuul pipelines and projects

Zuul pipelines define which Gerrit events Zuul listens to, how it reacts to those events and which output is returned to Gerrit.

Zuul projects define which Gerrit projects (repositories) zuul will monitor for new patches, and which Jenkins jobs to run for those projects.

Zuul projects and pipelines are configured in the “layout.conf” file. We can edit the file through the “zuul-server“:

sudo docker exec -it zuul-server vi /etc/zuul/layout.yaml

Here is the “layout.yaml file I wrote to work with the projects and jobs I’ve created:

- name: check-patch
  manager: IndependentPipelineManager
  source: gerrit
      - event: patchset-created
      - event: draft-published
      - event: change-restored
    gerrit: {}
      verified: 1
      verified: -1

- name: check-merged
  manager: DependentPipelineManager
  source: gerrit
      - event: comment-added
          - code-review: 2
    gerrit: {}
      verified: 1
      submit: true
      verified: -1
      code-review: -1

- name: zool-test
    - zuul-check-patch
    - zuul-check-merged

Things to note:

  • The “check-patch”  pipeline is triggered whenever a patch set is created (see lines #6 through #9)
  • The “check-patch” pipeline runs independently per patch, as specified by line #3.
  • The “check-patch” pipeline sends a message to Gerrit as it starts (lines #10 – #11), and switches the “verified” flag to “+1” or “-1” if the jobs succeed or fail respectively.
  • The “check-merged” pipeline is triggered when the “code-review” flag is set to “+2” on a Gerrit patch (line #26).
  • The “check-merged” pipeline takes patch dependencies into consideration (line #20)
  • The “check-merged” pipeline submits the patch for merging when jobs are successful (Line #32)
  • Both pipelines monitor patches for the “zuul-test” project (Line #39).
  • The “check-patch” pipeline runs the “zuul-check-patch” Jenkins job (Line #41).
  • The “check-merged” pipeline runs the “zuul-check-merged” Jenkins job (Line #42).

Once we alter the “layout.yaml” file, we need to make “zuul-server” reload it. we can do that with the following command:

sudo docker kill -s SIGHUP zuul-server

Everything should start playing together now – adding patches to the “zuul-test” project should make Zuul run the “zuul-check-patch” job in Jenkins. Setting the “code-review” flag to “+2” on patches will make Zuul run the “zuul-check-merged” job in Jenkins, and merge the patch if it succeeds.

We can examine how “zuul-merger” and the dependent pipeline work by setting “code-review” to “+2” on several patches at once. We should see jobs getting started for all patches at once while each patch code gets rebased on top of the patch that was set to “+2” before it was.

As Zuul triggers jobs, we can monitor the status of its pipelines by glancing at its graphical UI which would be exposed at


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s