Deploying Dragonfly on a Kubernetes Cluster

1. Deploy NFS Storage

1.1 Configure the NFS Service on the Storage Node

Install the NFS service on the storage node

1
apt-get install -y nfs-kernel-server

Start the NFS service and enable it at boot

1
systemctl start nfs-server.service & systemctl enable nfs-server.service

Configure the NFS shared directory

1
mkdir -p /data/nfs

1
vim /etc/exports

/data/nfs  *(rw,sync,no_root_squash,no_all_squash)

Restart the NFS service

1
systemctl restart nfs-server.service

1.2 Configure the NFS Client on All Nodes

1
apt-get install -y nfs-common

2 Configure StorageClass

1
2
helm repo add csi-driver-nfs https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/charts
helm install csi-driver-nfs csi-driver-nfs/csi-driver-nfs --namespace kube-system --version v4.9.0

1
export NFS_SERVER_IP=x.x.x.x

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
kubectl apply -f - <<EOF
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-csi-storageclass
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: nfs.csi.k8s.io
parameters:
  server: ${NFS_SERVER_IP}
  share: /data/nfs
reclaimPolicy: Delete
volumeBindingMode: Immediate
mountOptions:
  - nfsvers=4.1
EOF

3 Deploy Dragonfly

Add the Helm repository

1
2
helm repo add dragonfly https://dragonflyoss.github.io/helm-charts/
helm repo update

Install Dragonfly

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
cat <<EOF > dragonfly-values.yaml
manager:
  image:
    repository: dragonflyoss/manager
    tag: latest
  metrics:
    enable: true

scheduler:
  image:
    repository: dragonflyoss/scheduler
    tag: latest
  metrics:
    enable: true

seedClient:
  image:
    repository: dragonflyoss/client
    tag: latest
  metrics:
    enable: true

client:
  image:
    repository: dragonflyoss/client
    tag: latest
  metrics:
    enable: true
EOF

1
helm install --create-namespace --namespace dragonfly-system dragonfly dragonfly/dragonfly -f dragonfly-values.yaml

Uninstall

1
helm uninstall dragonfly --namespace dragonfly-system

Change the cache directory

1
kubectl -n dragonfly-system edit ds dragonfly-client

1
2
3
volumes:
  - emptyDir: {}
    name: storage

Change it to

1
2
3
4
5
volumes:
  - hostPath:
      path: /data/dragonfly
      type: DirectoryOrCreate
    name: storage

Change the client timeout configuration

1
kubectl -n dragonfly-system edit cm dragonfly-client

Adjust pieceTimeout under download to make downloading large image files easier.

1
kubectl -n dragonfly-system rollout restart ds dragonfly-client

4 Configure Clients to Use Dragonfly

In older versions the default proxy port is 65001, and gen-containerd-hosts.sh is used to generate the configuration file; in newer versions it is 4001, and gen-containerd-hosts2.sh generates the configuration file.

1
2
3
4
5
wget -O /tmp/gen-containerd-hosts.sh https://raw.githubusercontent.com/shaowenchen/demo/master/dragonfly/gen-containerd-hosts2.sh

bash /tmp/gen-containerd-hosts.sh docker.io
bash /tmp/gen-containerd-hosts.sh registry.cn-beijing.aliyuncs.com
bash /tmp/gen-containerd-hosts.sh registry.cn-shanghai.aliyuncs.com

1
ls /etc/containerd/certs.d/