How to Use Docker in Docker

1. Typical Use Cases

In CI, there is usually a CI Engine responsible for parsing the pipeline and controlling the whole build process, while the actual build is delegated to an Agent. Jenkins and GitLab both work this way.

As shown below, there are many kinds of Agents that connect to the CI Engine. This is to meet the varying build environment requirements of different projects.

At the same time, Agents are dynamic — needed only during a build, destroyed when the build finishes. CI is an excellent fit for practicing container, Serverless, and similar technologies, so in production the Agent is often containerized.

That raises the question: if the CI Engine is also containerized, how do you use an Agent container to build inside a container? And if the Agent is already containerized, how do you build images on the Agent? This article answers both — how to use Docker in Docker.

2. Two Usage Modes

We need to know that Docker works in C/S mode, mainly split into two parts: the Docker CLI and the Docker Daemon. The Docker CLI, i.e. the client, gives users command-line access to Docker, for example docker create/images/ps. The Docker Daemon, i.e. the daemon process, receives user commands and maintains the container lifecycle.

2.1 Docker in Docker

Docker in Docker, abbreviated DinD below.

As shown above, you can run a Docker Daemon directly inside a Container, then use the Docker CLI tool inside the Container to manage containers.

In this mode, the Docker Daemon inside the container is fully independent of the outside, with good isolation properties. It looks as though the Container is similar to a VM, but the author of DinD does not particularly recommend it either.

The main reason is still security. DinD has to be started in privileged mode, and this nesting brings potential security risks.

In this mode, the container responding to commands is nested inside the container using the docker command.

2.2 Docker outside of Docker

Docker outside of Docker, abbreviated DooD below.

As shown above, Docker works in C/S mode; in use, the user cares about the C side, while lifecycle management sits on the S side.

So you only need to mount the Container’s external Docker Daemon service into the Container. The Container is led to believe a Docker Daemon is running locally, and when you use Docker CLI commands, the external Docker Daemon responds to the request.

In this mode, the container responding to commands is at the same level as the container using the docker command.

3. Demonstration in a Docker Environment

3.1 DinD

Run the DinD container

1
2
3
docker run --security-opt apparmor=unconfined --security-opt seccomp=unconfined --privileged -e DOCKER_TLS_CERTDIR="" -d --name dockerd  docker:dind

d6414f2ff0076c42de19a8a1fe122481c1a72b3bd45fd490dbe1c427414b4139

Run a container with the CLI, linked to the DinD container

1
docker run --rm -it --link dockerd:docker docker:latest sh

Pull an image in the DinD container

Pull an image

1
docker pull shaowenchen/devops-java-sample

View the images

1
2
3
4
docker images

REPOSITORY                       TAG                 IMAGE ID            CREATED             SIZE
shaowenchen/devops-java-sample   latest              fa4651c24a18        6 weeks ago         122MB

It works just like an independent Docker Daemon environment.

Check whether the outside is affected

Type exit to leave the container, and go through the Docker Daemon on the host

1
docker images |grep fa4651c24a18

As expected. DinD uses an independent Docker Daemon and has no direct impact on the external instance.

3.2 DooD

Run a container

1
docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock alpine sh

Install curl

Here, to avoid installing the Docker CLI, we call the Docker Daemon’s API directly with curl.

1
apk update && apk add curl

Pull an image

1
2
3
4
curl -XPOST --unix-socket /var/run/docker.sock http://localhost/images/create?fromImage=shaowenchen/docker-robotframework&tag=latest

...
{"status":"Status: Downloaded newer image for shaowenchen/docker-robotframework"}

View the pulled image

Type exit to leave the container, and go through the Docker Daemon on the host

1
2
3
docker images |grep robotframework

shaowenchen/docker-robotframework                              latest                         d99cfa7ee716        12 months ago       1.5GB

As expected. DooD uses the external Docker Daemon directly.

4. Demonstration in a Kubernetes Environment

4.1 DinD

Create a dind.yaml file with the following content:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
apiVersion: apps/v1
kind: Deployment
metadata:
  name: dind
spec:
  replicas: 1
  selector:
    matchLabels:
      app: dind
  template:
    metadata:
      labels:
        app: dind
    spec:
      containers:
        - name: dockerd
          image: "docker:dind"
          env:
            - name: DOCKER_TLS_CERTDIR
              value: ""
          securityContext:
            privileged: true
        - name: docker-cli
          image: "docker:latest"
          env:
            - name: DOCKER_HOST
              value: 127.0.0.1
          command: ["/bin/sh"]
          args: ["-c", "sleep 86400;"]

Create the Deployment

1
kubectl apply -f dind.yaml

View the created Pod name

1
2
3
kubectl get pod |grep dind

dind-5446ffbc8d-68q28   2/2     Running       0          12s

Enter the Pod

1
kubectl exec -it dind-5446ffbc8d-68q28  -c docker-cli sh

Test whether it uses an independent Docker Daemon

1
docker pull nginx

1
2
3
4
docker images

REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
nginx               latest              daee903b4e43        3 days ago          133MB

As expected, only the Nginx image just pulled is shown here, completely independent of the host’s Docker Daemon.

4.2 DooD

Create a dood.yaml file with the following content:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
apiVersion: apps/v1
kind: Deployment
metadata:
  name: dood
spec:
  replicas: 1
  selector:
    matchLabels:
      app: dood
  template:
    metadata:
      labels:
        app: dood
    spec:
      containers:
        - image: docker:latest
          name: docker-cli
          securityContext:
            privileged: false
          command: ["/bin/sh"]
          args: ["-c", "sleep 86400;"]
          volumeMounts:
            - mountPath: /var/run/docker.sock
              name: volume-docker
      volumes:
        - hostPath:
            path: /var/run/docker.sock
            type: ""
          name: volume-docker

Create the Deployment

1
kubectl apply -f dood.yaml

View the created Pod name

1
2
3
kubectl get pod  |grep dood

dood-667d8bcfc6-d5fzf   1/1     Running   0          15s

Enter the Pod

1
kubectl exec -it dood-667d8bcfc6-d5fzf  -c docker-cli sh

Test whether it uses the host’s Docker Daemon

1
2
3
docker images |wc

69       482      8509

As expected. The Docker command here uses the external Docker Daemon.