Multi-Arch Docker Images

1. Enabling Docker’s experimental Feature

First, enable Docker’s experimental feature, so the commands below are available.

Edit vim ~/.docker/config.json and add the following:

1
2
3
4
{
  "experimental": "enabled",
  "debug": true
}

Note that this is not the /etc/docker/daemon.json file, and Docker does not need to be restarted.

2. Docker Images

Starting with Docker 1.10 and Registry 2.3, Docker introduced the manifest to describe image metadata.

2.1 How a Dockerfile Becomes an Image

As shown above, every command line in a Dockerfile is associated with a layer when the image is built. A layer is a simple wrapper around an image layer. When these image layers are stored they are reused — that is, when multiple images use the same image layer, only one copy is stored. There are some conceptual details here that we can ignore for now.

2.2 How Docker and Registry Transfer Images

Image metadata includes information such as size, digest, and layers.

When Docker and a Registry push or pull an image, they first transfer the manifest. Only when an image layer does not already exist in the current environment is it transferred over the network; otherwise the local image layer is reused directly.

2.3 The manifest File Structure

Inspect the manifest of an image directly.

Inspect the manifest

1
docker manifest inspect jenkins/jnlp-slave

Example output (JSON):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
{
	"schemaVersion": 2,
	"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
	"config": {
		"mediaType": "application/vnd.docker.container.image.v1+json",
		"size": 12327,
		"digest": "sha256:9b5976169d3504ea796a4af75f8648db9a500e8b9351ee19276031108c59f429"
	},
	"layers": [
		{
			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
			"size": 50382041,
			"digest": "sha256:f15005b0235fa8bd31cc6988c4f2758016fe412d696e81aecf73e52be079f19e"
		},
		{
			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
			"size": 7812166,
			"digest": "sha256:41ebfd3d2fd0de99b1c63aa36a507bf5555481d06e571d84ed84440d30671494"
		},
		...
  ]
}

Inspect a multi-arch manifest

Through the manifest.list mediaType, multiple images can be combined into one. Under different architecture and os conditions, Docker automatically pulls the image that fits the current environment.

1
docker manifest inspect maven

Example output (JSON):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 1579,
         "digest": "sha256:3c5d1c8795c96b775723cf912b297850feb1fc8f4b98ec2eda4303f3a6277310",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      }
   ]
}

3. Common Ways to Organize Multi-Arch Images

The images that different OSes and CPUs can run differ. At the OS level, images mainly fall into Windows and Linux-like images. At the CPU level, there are mainly amd64, arm, ppc64le, s390x and other architectures. amd64 is x86-64, and is usually the default architecture, so amd64 and linux do not need to be specified explicitly.

3.1 Distinguishing Architectures by namespace

Format namespaces-{ARCH}-{OS}/image:tag

amd64

1
shaowenchen-amd64/coredns:latest

1
shaowenchen-arm/coredns:latest

3.2 Distinguishing Architectures by image name

Format namespaces/image-{ARCH}-{OS}:tag

amd64

1
shaowenchen/coredns-amd64:latest

1
shaowenchen/coredns-arm:latest

3.3 Distinguishing Architectures by tag name

Format namespaces/image:tag-{ARCH}-{OS}

amd64

1
shaowenchen/coredns:latest-amd64

1
shaowenchen/coredns:latest-arm

4. Managing Multi-Arch Images with manifest list

A manifest list is a list of image manifests, used to hold image information for different architectures. Put simply, it creates a new entry point for pulling images and associates images of different architectures with it, without generating any new image layers.

First, push the images

1
2
3
docker push shaowenchen/coredns:coredns-amd64
docker push shaowenchen/coredns:coredns-arm
docker push shaowenchen/coredns:coredns-arm64

Otherwise, the next step fails with no such manifest: docker.io/shaowenchen/coredns:coredns-amd64

Create the multi-arch manifest list

1
2
3
4
5
6
docker manifest create shaowenchen/coredns:latest \
    shaowenchen/coredns:coredns-amd64 \
    shaowenchen/coredns:coredns-arm \
    shaowenchen/coredns:coredns-arm64 --amend

Created manifest list docker.io/shaowenchen/coredns:latest

[Optional] Update the information for the relevant architecture

1
docker manifest annotate shaowenchen/coredns:latest shaowenchen/coredns:coredns-arm --arch arm

Inspect the manifest list

1
docker manifest inspect shaowenchen/coredns:latest

Example output (JSON):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 739,
         "digest": "sha256:242d440e3192ffbcecd40e9536891f4d9be46a650363f3a004497c2070f96f5a",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 739,
         "digest": "sha256:e9e08bce9d74a48723518a476f67ada25d00ed69dd3719c3fde41b10d390b0d0",
         "platform": {
            "architecture": "arm",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 739,
         "digest": "sha256:969a21696cff473cb4d36854b15118885fd414394d09444cfd111213fabcd982",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      }
   ]
}

Push to DockerHub

1
2
3
docker manifest push shaowenchen/coredns:latest

sha256:bcdaff4935b922edd8f415323e04d3a77374f5ddabe0a59d649fd0b6be4ad5ef

Check the pushed image on the DockerHub page

On any architecture, the same single command pulls the image:

1
docker pull shaowenchen/coredns:latest

This brings great convenience to deployment — there is no need to append architecture and os information to the image. The same deployment program can be used across architectures.

5. References

https://docs.docker.com/engine/reference/commandline/manifest/