Kubernetes Volumes

1. Docker Storage Volumes

1.2 Volumes in Docker

A Docker volume mounts a host directory into a container. Files modified inside the container are persisted to the host. Even if the container is deleted, the files on the host are retained.

Docker uses /var/lib/docker/volumes/ to store container volumes.

List local volumes:

1
2
3
4
5
6
7
8
9
tree /var/lib/docker/volumes/ -L 3
/var/lib/docker/volumes/
|-- 714450f353b26b5aa57aa352766c201c0851685e0e28c2e67ae1631f29c465b4
|   `-- _data
|       |-- access.log -> /dev/stdout
|       `-- error.log -> /dev/stderr
|-- metadata.db
`-- volume_name
    `-- _data

When a volume is created, Docker creates a directory under /var/lib/docker/volumes/ to store the data. After the container is stopped or destroyed, the data in the volume still persists.

1.2 Docker Volume Operations

Create a volume:

1
docker volume create volume_name

List all volumes:

1
docker volume ls

Inspect a volume:

1
docker volume inspect volume_name

Remove a volume:

1
docker volume rm volume_name

Mount a volume into a container:

1
docker run -it -v volume_name:/data centos /bin/bash

2. Storage Volumes in Kubernetes

2.1 Kubernetes Volumes

Every node has a container runtime; here we take the Docker runtime as an example. Mounting the way Docker volumes do runs into a problem: because Kubernetes schedules Pods freely, containers in a Pod have no binding relationship with a node, so there is no guarantee the volume can always be mounted successfully, nor that the integrity and consistency of the data can be guaranteed.

The solution is quite conventional: use a service to provide stateful capability. Kubernetes offers PVs and PVCs as the way to use volumes.

For a standalone storage backend, the implementation can be NFS, Ceph, GlusterFS, and so on. A PV can carve out a portion of it for Kubernetes storage, and its lifecycle does not depend on the Pod. Containers are the real users of volumes, and every container in a Pod must specify the mount location for each volume.

To use a volume, you need to specify it for the Pod (the spec.volumes field) as well as the location where it is mounted into the container (the spec.containers.volumeMounts field).

2.2 Storage Plugins

Storage plugins are used in Kubernetes to provide volume support. Kubernetes storage plugins fall into:

in-tree

in-tree plugins run inside Kubernetes core components. When the corresponding volume service is needed, the plugin in the core component is called.

Volume types natively supported by Kubernetes:

GCEPersistentDisk, AWSElasticBlockStore, AzureFile, AzureDisk, FC (Fibre Channel), FlexVolume, Flocker, NFS, iSCSI, CephFS, Cinder (OpenStack block storage), Glusterfs, VsphereVolume, Quobyte Volumes, etc.

out-of-tree

out-of-tree plugins have their code and deployment independent of Kubernetes. They are usually used to supplement storage types that in-tree does not support, or to customize and extend storage functionality.

Starting with version 1.8, the Kubernetes Storage SIG stopped accepting in-tree plugins and recommends that all storage providers use out-of-tree plugins. There are currently two recommended implementation approaches: the Container Storage Interface (CSI) and Flexvolume.

2.3 PV and PVC

PV is short for PersistentVolume, and PVC is short for PersistentVolumeClaim.

PV and PVC are two resources provided by Kubernetes, and users can operate on them through the API.

Administrators only need to focus on how to provide storage through PVs, not on how users consume it.

Users only need to focus on how to mount a PVC into a container, not on how the storage volume is implemented.

Using a volume usually breaks down into the following steps:

Create a Persistent Volume
Create a Persistent Volume Claim
Create a Pod and use the PVC

3. References

https://jimmysong.io/posts/kubernetes-volumes-introduction/