Jenkins is a free and open source automation server, which is used to automate software building, testing, deployment, etc.

I wanted to have a quick and easy way to run Jenkins inside docker, but also use docker containers to run jobs on the dockerized Jenkins. Using docker for jobs makes it easy to encode job runtime dependencies in the source code repo itself.

The official document on running Jenkins in docker is pretty comprehensive. But, I wanted a version using docker-compose (on Linux).

So, I started with a basic compose file:

version: '3.7'
services:
  jenkins:
  	image: jenkins/jenkins:alpine
    ports:
      - 8081:8080
    container_name: jenkins
    volumes:
      - ./home:/var/jenkins_home

When using this ( docker-compose up -d ), things came up properly, but Jenkins did not have access to the docker daemon running on the host. Also, the docker cli binary is not present inside the container.

The way to achieve this was to mount the docker socket and cli binary to inside the container so that it can be accessed. So, we come to the following compose file:

version: '3.7'
services:
  jenkins:
    image: jenkins/jenkins:alpine
    ports:
      - 8081:8080
    container_name: jenkins
    volumes:
      - ./home:/var/jenkins_home
      - /var/run/docker.sock:/var/run/docker.sock
      - /usr/bin/docker:/usr/local/bin/docker

But, when trying to run docker ps inside the container with the above compose file, I was still getting the error: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock. This is because the Jenkins container is running with the jenkins user, which does not have access to use that socket.

From my research, the commonly recommended ways to solve this problem were:

• Run the container as root user
• chmod the socket file to 777
• Install sudo inside the container and give the jenkins user access to sudo without needing to enter password.

A more secure way is to create the docker group inside the container, and add the jenkins user to that group. But, this requires us to build a custom image.

Also, the group id of the docker group inside and outside the container have to be the same, so I had to add an extra check which deletes any existing group inside the container which uses the same group id, then creates the new docker group with the passed group id, and then adds the jenkins user to the docker group.

So, the final Dockerfile is:

FROM jenkins/jenkins:alpine
ARG docker_group_id=999

USER root
RUN old_group=$(getent group $docker_group_id | cut -d: -f1) && \
    ([ -z "$old_group" ] || delgroup "$old_group") && \
    addgroup -g $docker_group_id docker && \
    addgroup jenkins docker

USER jenkins

And the final docker-compose.yml file is:

version: '3.7'
services:
  jenkins:
    build:
      context: .
      args:
        docker_group_id: 999
    ports:
      - 8081:8080
    container_name: jenkins
    volumes:
      - ./home:/var/jenkins_home
      - /var/run/docker.sock:/var/run/docker.sock
      - /usr/bin/docker:/usr/local/bin/docker

The docker_group_id argument can be edited in the compose file. Command to get the group id of docker:

$ getent group docker | cut -d: -f3

With the above, everything works:

$ docker-compose up -d
Creating network "jenkins_test_default" with the default driver
Building jenkins
Step 1/6 : FROM jenkins/jenkins:alpine
alpine: Pulling from jenkins/jenkins
801bfaa63ef2: Pull complete
2b72e22c6786: Pull complete
8d16efe80b55: Pull complete
682cd8857a9a: Pull complete
29c6010e8988: Pull complete
fa466f5d199d: Pull complete
e047245de0ff: Pull complete
0cfb53380af7: Pull complete
c29612b1a095: Pull complete
cd7d4bd47719: Pull complete
21cd3d960a1f: Pull complete
f3962370d584: Pull complete
bd6f35a1ea17: Pull complete
bd0c271b250f: Pull complete
Digest: sha256:1c3d9a1ed55911f9b165dd122118bff5da57520effb180d36b5c19d2a0cfe645
Status: Downloaded newer image for jenkins/jenkins:alpine
 ---> e14be04b79e8
Step 2/6 : ARG docker_group_id=999
 ---> Running in f1922fa97177
Removing intermediate container f1922fa97177
 ---> 79460069fb98
Step 3/6 : RUN echo "Assuming docker group id: $docker_group_id"
 ---> Running in 11809f4ae767
Assuming docker group id: 999
Removing intermediate container 11809f4ae767
 ---> e89b345f6c74
Step 4/6 : USER root
 ---> Running in b2e311372bc9
Removing intermediate container b2e311372bc9
 ---> 9d4d8c3ad5b2
Step 5/6 : RUN old_group=$(getent group $docker_group_id | cut -d: -f1) &&     ([ -z "$old_group" ] || delgroup "$old_group") &&     addgroup -g $docker_group_id docker &&     addgroup jenkins docker
 ---> Running in 357046a8ac49
Removing intermediate container 357046a8ac49
 ---> 865b942324eb
Step 6/6 : USER jenkins
 ---> Running in dbc2976f62c0
Removing intermediate container dbc2976f62c0
 ---> c7e6fac0187c

Successfully built c7e6fac0187c
Successfully tagged jenkins_test_jenkins:latest
WARNING: Image for service jenkins was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Creating jenkins ... done

$ docker-compose exec jenkins docker ps
CONTAINER ID   IMAGE                  COMMAND                  CREATED          STATUS          PORTS                               NAMES
6c05ee1315e4   jenkins_test_jenkins   "/sbin/tini -- /usr/…"   47 seconds ago   Up 47 seconds   50000/tcp, 0.0.0.0:8081->8080/tcp   jenkins

Next Steps

Here is an excellent guide on how to setup Jenkins configuration as code. This will make this setup even better because nothing will need to be configured inside Jenkins manually - it will all be driven by code / files.

Updated for 2023: I've updated this post with the following changes:

1. Added a top-level sample playbook
2. Used ansible apt_module's cache_time parameter to prevent repeated apt-get updates
3. Install docker-compose-plugin using apt (provides docker compose v2)
4. Make installing docker compose v1 optional
5. Various fixes as suggested in comments
6. Tested against Debian 10,11,12 and Ubuntu 18.04 (bionic), 20.04 (focal), 22.04 (jammy) using Vagrant.

I've published a new post on how I've done this testing.

I wanted a simple, but optimal (and fast) way to install docker and docker-compose using Ansible. I found a few ways online, but I was not satisfied.

My requirements were:

• Support Debian and Ubuntu
• Install docker and docker compose v2 using apt repositories
• Prevent unnecessary apt-get update if it has been run recently (to make it fast)
• Optionally install docker compose v1 by downloading from github releases
  • But, don’t download if current version >= the minimum version required

I feel trying to achieve these requirements gave me a very good idea of how powerful ansible can be.

The final role and vars files can be seen in this gist. But, I’ll go through each section below to explain what makes this better / faster.

File structure

playbook.yml
roles/
├── docker/
│    ├── defaults/
│    │   ├── main.yml
│    ├── tasks/
│    │   ├── main.yml
│    │   ├── docker_setup.yml

Playbook

This is the top-level playbook. Any default vars mentioned below can be overridden here.

---
- hosts: all
  vars:
    - docker_compose_install_v1: true
    - docker_compose_version_v1: "1.29.2"
  tasks:
    - name: Docker setup
      block:
        - import_role: name=docker

Variables

First, we’ve defined some variables in defaults/main.yml. These will control which release channel of docker will be used and whether to install docker compose v1.

---
docker_apt_release_channel: stable
docker_apt_arch: amd64
docker_apt_repository: "deb [arch={{ docker_apt_arch }}] https://download.docker.com/linux/{{ ansible_distribution | lower }} {{ ansible_distribution_release }} {{ docker_apt_release_channel }}"
docker_apt_gpg_key: https://download.docker.com/linux/{{ ansible_distribution | lower }}/gpg
docker_compose_install_v1: false
docker_compose_version_v1: "1.29.2"

Role main.yml

The tasks/main.yml file imports tasks from tasks/docker_setup.yml and turns on become for the whole task.

---
- import_tasks: docker_setup.yml
  become: true

Docker Setup

This task is divided into the following sections:

Install dependencies

- name: Install packages using apt
  apt:
    name: 
        - apt-transport-https
        - ca-certificates
        - curl
        - gnupg2
        - software-properties-common
    state: present
    cache_valid_time: 86400

Here the state: present makes sure that these packages are only installed if not already installed. I've set cache_valid_time to 1 day so that apt-get update is not run if it has already run recently.

Add docker repository

- name: Add Docker GPG apt Key
  apt_key:
    url: "{{ docker_apt_gpg_key }}"
    state: present

- name: Add Docker Repository
  apt_repository:
    repo: "{{ docker_apt_repository }}"
    state: present
    update_cache: true

Here, the state: present and update_cache: true make sure that the cache is only updated if this state was changed. So, apt-get update is not run if the docker repo is already present.

Install and enable docker and docker compose v2

- name: Install docker-ce
  apt:
    name: docker-ce
    state: present
    cache_valid_time: 86400

- name: Run and enable docker
  service:
    name: docker
    state: started
    enabled: true

- name: Install docker compose
  apt:
    name: docker-compose-plugin
    state: present
    cache_valid_time: 86400

Again, due to state: present and cache_valid_time: 86400, there are no extra cache fetches if docker and docker-compose-plugin are already installed.

Docker Compose V1 Setup

WARNING: docker-compose v1 is end-of-life, please keep that in mind and only install/use it if absolutely required.

This task is wrapped in an ansible block that checks if docker_compose_install_v1 is true.

- name: Install docker-compose v1
  when:
    - docker_compose_install_v1 is defined
    - docker_compose_install_v1
  block:

Inside the block, there are two sections:

Check if docker-compose is installed and it’s version

- name: Check current docker-compose version
  command: docker-compose --version
  register: docker_compose_vsn
  changed_when: false
  failed_when: false
  check_mode: no

- set_fact:
    docker_compose_current_version: "{{ docker_compose_vsn.stdout | regex_search('(\\d+(\\.\\d+)+)') }}"
  when:
    - docker_compose_vsn.stdout is defined

The first block saves the output of docker-compose --version into a variable docker_compose_vsn. The failed_when: false ensures that this does not call a failure even if the command fails to execute. (See error handling in ansible).

Sample output when docker-compose is installed: docker-compose version 1.26.0, build d4451659

The second block parses this output and extracts the version number using a regex (see ansible filters). There is a when condition which causes the second block to skip execution if the first block failed (See playbook conditionals).

Install or upgrade docker-compose if required

- name: Install or upgrade docker-compose
  get_url: 
    url : "https://github.com/docker/compose/releases/download/{{ docker_compose_version }}/docker-compose-Linux-x86_64"
    dest: /usr/local/bin/docker-compose
    mode: 'a+x'
    force: yes
  when: >
    docker_compose_current_version == ""
    or docker_compose_current_version is version(docker_compose_version, '<')

This just downloads the required docker-compose binary and saves it to /usr/local/bin/docker-compose, but it has a conditional that this will only be done if either docker-compose is not already installed, or if the installed version is less than the required version. To do version comparison, it uses ansible’s built-in version comparison function.

So, we used a few ansible features to achieve what we wanted. I’m sure there are a lot of other things we can do to make this even better and more fool-proof. Maybe a post for another day.