Running docker jobs inside Jenkins running on docker

Jenkins is a free and open source automation server, which is used to automate software building, testing, deployment, etc.

I wanted to have a quick and easy way to run Jenkins inside docker, but also use docker containers to run jobs on the dockerized Jenkins. Using docker for jobs makes it easy to encode job runtime dependencies in the source code repo itself.

The official document on running Jenkins in docker is pretty comprehensive. But, I wanted a version using docker-compose (on Linux).

So, I started with a basic compose file:

version: '3.7'
services:
  jenkins:
  	image: jenkins/jenkins:alpine
    ports:
      - 8081:8080
    container_name: jenkins
    volumes:
      - ./home:/var/jenkins_home

docker-compose.yml

When using this ( docker-compose up -d ), things came up properly, but Jenkins did not have access to the docker daemon running on the host. Also, the docker cli binary is not present inside the container.

The way to achieve this was to mount the docker socket and cli binary to inside the container so that it can be accessed. So, we come to the following compose file:

version: '3.7'
services:
  jenkins:
    image: jenkins/jenkins:alpine
    ports:
      - 8081:8080
    container_name: jenkins
    volumes:
      - ./home:/var/jenkins_home
      - /var/run/docker.sock:/var/run/docker.sock
      - /usr/bin/docker:/usr/local/bin/docker

docker-compose.yml

But, when trying to run docker ps inside the container with the above compose file, I was still getting the error: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock. This is because the Jenkins container is running with the jenkins user, which does not have access to use that socket.

From my research, the commonly recommended ways to solve this problem were:

Run the container as root user
chmod the socket file to 777
Install sudo inside the container and give the jenkins user access to sudo without needing to enter password.

A more secure way is to create the docker group inside the container, and add the jenkins user to that group. But, this requires us to build a custom image.

Also, the group id of the docker group inside and outside the container have to be the same, so I had to add an extra check which deletes any existing group inside the container which uses the same group id, then creates the new docker group with the passed group id, and then adds the jenkins user to the docker group.

So, the final Dockerfile is:

FROM jenkins/jenkins:alpine
ARG docker_group_id=999

USER root
RUN old_group=$(getent group $docker_group_id | cut -d: -f1) && \
    ([ -z "$old_group" ] || delgroup "$old_group") && \
    addgroup -g $docker_group_id docker && \
    addgroup jenkins docker

USER jenkins

Dockerfile

And the final docker-compose.yml file is:

version: '3.7'
services:
  jenkins:
    build:
      context: .
      args:
        docker_group_id: 999
    ports:
      - 8081:8080
    container_name: jenkins
    volumes:
      - ./home:/var/jenkins_home
      - /var/run/docker.sock:/var/run/docker.sock
      - /usr/bin/docker:/usr/local/bin/docker

docker-compose.yml

The docker_group_id argument can be edited in the compose file. Command to get the group id of docker:

$ getent group docker | cut -d: -f3

With the above, everything works:

$ docker-compose up -d
Creating network "jenkins_test_default" with the default driver
Building jenkins
Step 1/6 : FROM jenkins/jenkins:alpine
alpine: Pulling from jenkins/jenkins
801bfaa63ef2: Pull complete
2b72e22c6786: Pull complete
8d16efe80b55: Pull complete
682cd8857a9a: Pull complete
29c6010e8988: Pull complete
fa466f5d199d: Pull complete
e047245de0ff: Pull complete
0cfb53380af7: Pull complete
c29612b1a095: Pull complete
cd7d4bd47719: Pull complete
21cd3d960a1f: Pull complete
f3962370d584: Pull complete
bd6f35a1ea17: Pull complete
bd0c271b250f: Pull complete
Digest: sha256:1c3d9a1ed55911f9b165dd122118bff5da57520effb180d36b5c19d2a0cfe645
Status: Downloaded newer image for jenkins/jenkins:alpine
 ---> e14be04b79e8
Step 2/6 : ARG docker_group_id=999
 ---> Running in f1922fa97177
Removing intermediate container f1922fa97177
 ---> 79460069fb98
Step 3/6 : RUN echo "Assuming docker group id: $docker_group_id"
 ---> Running in 11809f4ae767
Assuming docker group id: 999
Removing intermediate container 11809f4ae767
 ---> e89b345f6c74
Step 4/6 : USER root
 ---> Running in b2e311372bc9
Removing intermediate container b2e311372bc9
 ---> 9d4d8c3ad5b2
Step 5/6 : RUN old_group=$(getent group $docker_group_id | cut -d: -f1) &&     ([ -z "$old_group" ] || delgroup "$old_group") &&     addgroup -g $docker_group_id docker &&     addgroup jenkins docker
 ---> Running in 357046a8ac49
Removing intermediate container 357046a8ac49
 ---> 865b942324eb
Step 6/6 : USER jenkins
 ---> Running in dbc2976f62c0
Removing intermediate container dbc2976f62c0
 ---> c7e6fac0187c

Successfully built c7e6fac0187c
Successfully tagged jenkins_test_jenkins:latest
WARNING: Image for service jenkins was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Creating jenkins ... done

$ docker-compose exec jenkins docker ps
CONTAINER ID   IMAGE                  COMMAND                  CREATED          STATUS          PORTS                               NAMES
6c05ee1315e4   jenkins_test_jenkins   "/sbin/tini -- /usr/…"   47 seconds ago   Up 47 seconds   50000/tcp, 0.0.0.0:8081->8080/tcp   jenkins

Next Steps

Here is an excellent guide on how to setup Jenkins configuration as code. This will make this setup even better because nothing will need to be configured inside Jenkins manually - it will all be driven by code / files.

Running docker jobs inside Jenkins running on docker

Next Steps

Interactions