CI tools testing lab: Setting up Zuul Server

In this post I’m going to describe the various components of Zuul, and how to run zuul-server in a Docker container  in a lab setup.

This post is the third in a series of posts describing how I’ve used Docker to build a lab setup on my laptop to try out Zuul and check out its various features. Previously I’ve written about how I’ve started Jenkins and Gerrit to have the necessary services for Zuul to work against.

In order to gain a deep understanding of Zuul and prepare fur running it in production, I came up with a rather complex container layout . If you just want to see what Zuul looks like you can get a single container with everything (including Jenkins and Gerrit) here.

A fully functional Zuul system consists of several components:

  1. zuul-server – The main server listening for events from Gerrit, managing the task queues and launching jobs.
  2. Gearman – The main communication layer between all the system components. It can run as a sub-process of zuul-server.
  3. zuul-merger – A service for merging commits together and preparing code to be checked by CI jobs.
  4. HTTP server to make the Git repositories created by zuul-merger available the be cloned by jobs.
  5. Zuul status page – A JavaScript page to show a graphical representation of the Zuul queue status. One needs an HTTP server to expose it.

Apart from Gearman, which is managed by the zuul-server process and hence has to reside in the same container, I’ve decided to run each component in its own container.

The zuul_base container

zuul-server and zuul-merger are packaged together in the Zuul Python package. For efficiency, I decided to create a base container with it upon which the zuul-server and zuul-merger containers will be based.

Here is the “Dockerfile” for building the container:

FROM openshift/python-27-centos7
MAINTAINER Barak Korren
RUN scl enable python27 -- pip install --user zuul
USER root
RUN install -o default -g root -m 755 -d /var/lib/zuul
USER default
VOLUME ["/etc/zuul", "/var/lib/zuul"]

I based the container on the Python container from OpenShift.

I export the volumes for storing Zuul’s configuration and it running state. Those are needed both by zuul-server and zuul-merger.

To build the container image the following command could be run:

sudo docker build -t zuul_base:latest .

The “Dockerfile” above can be downloaded from GitHub.

The zuul-server container

I want all the Zuul configuration and state to be persistent and allowed to be edited manually. Therefore, I created the zuul-server container in such a way that it is stored on mounted data volumes. However, I did want the container to generate stub configuration files if they are not yet present in the persistent volume.

Here is the stub file for “/etc/zuul.conf”, the main configuration file for Zuul:

[zuul]
layout_config=/etc/zuul/layout.yaml

[gearman_server]
start=true

[gearman]
server=172.17.0.4

[connection gerrit]
driver=gerrit
server=172.17.0.3
port=29418
baseurl=http://172.17.0.3:8080/
user=zuul
sshkey=/etc/zuul/ssh/id_rsa

To save myself some time I’ve hard-coded the addresses for the containers I’m running in the file. As I move this to production, I will probably have to parametrize all the hard-coded values.

I also stubbed the Zuul layout file, which determines the pipelines and jobs Zuul will manage. This is just a minimal stub designed just to allow zuul-server to start up smoothly.

pipelines: []
projects: []

To place the configuration files on the persistent volume, I wrote the following start-up script for the container:

#!/bin/bash -e
mkdir -p '/etc/zuul'

if [[ ! -e '/etc/zuul/zuul.conf' ]]; then
  cp '/usr/share/zuul/zuul.conf.template' '/etc/zuul/zuul.conf'
fi

if [[ ! -e '/etc/zuul/layout.yaml' ]]; then
  cp '/usr/share/zuul/layout.yaml.template' '/etc/zuul/layout.yaml'
fi

if [[ ! -d '/etc/zuul/ssh' ]]; then
  install -m 700 -d '/etc/zuul/ssh'
fi

if [[ ! -e '/etc/zuul/ssh/id_rsa' ]]; then
  ssh-keygen -b 2048 -t rsa -N '' -f '/etc/zuul/ssh/id_rsa'
fi

if [[ ! -e "$HOME/.ssh/known_hosts" ]]; then
  install -m 700 -d "$HOME/.ssh"
  ln -s '/etc/zuul/ssh/known_hosts' "$HOME/.ssh/known_hosts"
fi

exec zuul-server -d

This scrip also generates an SSH key pair on the persistent volume, to be used by Zuul to connect to Gerrit.

Finally, here is the “Dockerfile” to build th zuul-server container:

FROM zuul_base:latest
MAINTAINER Barak Korren

EXPOSE 8001
EXPOSE 4730

CMD start_zuul_server.sh

ADD zuul.conf.template /usr/share/zuul/zuul.conf.template
ADD layout.yaml.template /usr/share/zuul/layout.yaml.template
ADD start_zuul_server.sh /usr/local/bin

I’m exposing both port 8001 which is used to query Zuul`s status, and port 4730 which is used by Gearman.

The various files mentioned above can be downloaded from GitHub.

The command used to build the container is as follows:

sudo docker build -t zuul_server:latest zuul_server/

To run the container I wrote the following script, similar to others we’ve seen in previous posts:

#!/bin/bash -x
DIRS=(/tmp/zuul/conf /tmp/zuul-server/state)
for dir in "${DIRS[@]}"; do
  [[ -d "$dir" ]] && continue
  sudo install -o 1001 -g 0 -d "$dir"
done
sudo docker run -d \
  -v /tmp/zuul/conf:/etc/zuul:z \
  -v /tmp/zuul-server/state:/var/lib/zuul:Z \
  --name zuul-server \
  zuul_server:latest
sudo docker inspect \
  -f $'Container running at: {{.NetworkSettings.IPAddress}} With ports: {{range $k, $v := .NetworkSettings.Ports}}{{$k}} {{end}}' \
  zuul-server

One thing to note is that I’m going to share the configuration directory with the zuul-merger container. This is why it is mounted with SElinux settings that allow it to be shared (lower case “z”).

Once the container is up and running we can look at its logs with the following command:

sudo docker logs -tf zuul-server

The logs should tell us that Zuul is failing to connect to Gerrit at this point, to remedy this we need to go to the Gerrit server we’ve previously set up, and add a “zuul” user. The SSH public key for that use would be given by this command:

sudo docker exec -it zuul-server cat /etc/zuul/ssh/id_rsa.pub

 

We can use “curl” to inspect the Zuul server’s queue status:

curl http://172.17.0.4:8001/status

There is probably not much to be seen in the curl output at this point, there will be once we get around to creating some queues and Jenkins jobs.

We still have some work cut out for us before we can have Zuul fully up and running. I’ll cover it in the next instalments to come.

 

 

 

 

 

 

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s