Please enable Javascript to view the contents

Fluid 使用 NFS Runtime 以及性能测试

 ·  ☕ 3 分钟

1. 创建 Dataset

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
kubectl apply -f - <<EOF
apiVersion: data.fluid.io/v1alpha1
kind: Dataset
metadata:
  name: nfs-demo
spec:
  mounts:
  - mountPoint: x.x.x.x:/x-x/
    name: nfs-demo
EOF

2. 创建 Runtime

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
kubectl apply -f - <<EOF
apiVersion: data.fluid.io/v1alpha1
kind: ThinRuntimeProfile
metadata:
  name: nfs
spec:
  fileSystemType: nfs
  fuse:
    image: fluidcloudnative/nfs
    imageTag: v0.1
    imagePullPolicy: IfNotPresent
    command:
      - "/usr/local/bin/entrypoint.sh"
EOF
1
2
3
4
5
6
7
8
kubectl apply -f - <<EOF
apiVersion: data.fluid.io/v1alpha1
kind: ThinRuntime
metadata:
  name: nfs-demo
spec:
  profileName: nfs
EOF

3. 创建测试 Pod

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: nfs-demo
spec:
  containers:
    - name: nfs-demo
      image: shaowenchen/demo-ubuntu
      volumeMounts:
        - mountPath: /data
          name: nfs-demo
  volumes:
    - name: nfs-demo
      persistentVolumeClaim:
        claimName: nfs-demo
EOF

4. 性能测试

4.1 测试工具

  • 安装 Opscli 工具
1
curl -sfL https://raw.githubusercontent.com/shaowenchen/ops/main/getcli.sh | VERSION=latest sh -
  • 测试命令
1
opscli task --filepath  ~/.ops/tasks/get-diskio-byfio.yaml --filename ./testfile --size 10g -i 127.0.0.1

4.2 三种 nfs 挂载方式

  1. Direct 直接挂载到主机目录
1
mount -t nfs x.x.x.x:/x-x/ /data/
  1. 使用 CSI-Driver-NFS 提供 NFS 存储

参考 https://github.com/kubernetes-csi/csi-driver-nfs

  1. 使用 Fluid NFS Runtime 提供 NFS 存储

4.3 测试结果 - 10 GB 数据

  • Random Read Testing (随机读取测试)
测试环境IOPS带宽 (MiB/s)用时 (msec)数据大小 (GiB)
Direct28.0k1139046910
CSI NFS Pod311512.284136810
Fluid NFS Pod28.2k1109305210
  • Random Write Testing (随机写入测试)
测试环境IOPS带宽 (MiB/s)用时 (msec)数据大小 (GiB)
Direct28.6k1129177510
CSI NFS Pod321712.681471810
Fluid NFS Pod29.2k1148967810
  • Sequential Read Testing (顺序读取测试)
测试环境IOPS带宽 (MiB/s)用时 (msec)数据大小 (GiB)
Direct30293792704310
CSI NFS Pod11501447122610
Fluid NFS Pod26063263142810
  • Sequential Write Testing (顺序写入测试)
测试环境IOPS带宽 (MiB/s)用时 (msec)数据大小 (GiB)
Direct30923872648910
CSI NFS Pod11541447094610
Fluid NFS Pod23002883560510
  • Random Read IOPS Testing (随机读取 IOPS 测试)
测试环境IOPS带宽 (MiB/s)用时 (msec)数据大小 (GiB)
Direct10.5k40.024981710
CSI NFS Pod310712.184348410
Fluid NFS Pod920235.928485210
  • Random Write IOPS Testing (随机写入 IOPS 测试)
测试环境IOPS带宽 (MiB/s)用时 (msec)数据大小 (GiB)
Direct14.2k55.618420510
CSI NFS Pod314812.383258910
Fluid NFS Pod12.0k50.620232110

4.4 测试结果 - 1 GB 数据

  • Random Read Testing (随机读取测试)
测试环境IOPS带宽 (MiB/s)用时 (msec)数据大小 (MiB)
Direct27.2k10696481024
CSI NFS Pod20.3k79.4129011024
Fluid NFS Pod36.1k14172551024
  • Random Write Testing (随机写入测试)
测试环境IOPS带宽 (MiB/s)用时 (msec)数据大小 (MiB)
Direct27.6k10894891024
CSI NFS Pod28.3k11092751024
Fluid NFS Pod20.5k80.3127591024
  • Sequential Read Testing (顺序读取测试)
测试环境IOPS带宽 (MiB/s)用时 (msec)数据大小 (MiB)
Direct429153619091024
CSI NFS Pod112514172801024
Fluid NFS Pod466258317571024
  • Sequential Write Testing (顺序写入测试)
测试环境IOPS带宽 (MiB/s)用时 (msec)数据大小 (MiB)
Direct468958617471024
CSI NFS Pod116114570521024
Fluid NFS Pod380347521541024
  • Random Read IOPS Testing (随机读取 IOPS 测试)
测试环境IOPS带宽 (MiB/s)用时 (msec)数据大小 (MiB)
Direct15.5k60.7168711024
CSI NFS Pod14.4k56.3181951024
Fluid NFS Pod15.0k58.8174191024
  • Random Write IOPS Testing (随机写入 IOPS 测试)
测试环境IOPS带宽 (MiB/s)用时 (msec)数据大小 (MiB)
Direct13.1k51.1200201024
CSI NFS Pod15.0k62.4164231024
Fluid NFS Pod13.2k51.5198751024

4.5 测试结果 - 100 MB 数据

好的,我将按照您提供的格式整理全部的测试案例。

  • Rand_Read_Testing (随机读取测试)
测试环境IOPS带宽 (MiB/s)用时 (msec)数据大小 (MiB)
Direct14.3k55.71794100
CSI NFS Pod28.1k110912100
Fluid NFS Pod37.0k145691100
  • Rand_Write_Testing (随机写入测试)
测试环境IOPS带宽 (MiB/s)用时 (msec)数据大小 (MiB)
Direct26.5k104966100
CSI NFS Pod29.4k115872100
Fluid NFS Pod905235.42828100
  • Sequ_Read_Testing (顺序读取测试)
测试环境IOPS带宽 (MiB/s)用时 (msec)数据大小 (MiB)
Direct3603450222100
CSI NFS Pod1568196510100
Fluid NFS Pod6779847118100
  • Sequ_Write_Testing (顺序写入测试)
测试环境IOPS带宽 (MiB/s)用时 (msec)数据大小 (MiB)
Direct6666833120100
CSI NFS Pod6349794126100
Fluid NFS Pod6451806124100
  • Rand_Read_IOPS_Testing (随机读取 IOPS 测试)
测试环境IOPS带宽 (MiB/s)用时 (msec)数据大小 (MiB)
Direct15.4k60.01666100
CSI NFS Pod14.6k56.81759100
Fluid NFS Pod15.4k60.21660100
  • Rand_Write_IOPS_Testing (随机写入 IOPS 测试)
测试环境IOPS带宽 (MiB/s)用时 (msec)数据大小 (MiB)
Direct13.5k52.61900100
CSI NFS Pod15.8k61.91616100
Fluid NFS Pod13.2k51.41944100
  • Rand_Read_Latency_Testing (随机读取延迟测试)
测试环境IOPS带宽 (kB/s)用时 (msec)数据大小 (MiB)
Direct496198551594100
CSI NFS Pod579231944159100
Fluid NFS Pod458183355868100
  • Rand_Write_Latency_Testing (随机写入延迟测试)
测试环境IOPS带宽 (kB/s)用时 (msec)数据大小 (MiB)
Direct458183655785100
CSI NFS Pod575230144495100
Fluid NFS Pod444177657651100

4.5 结论

  • Direct

直接挂载 NFS 的方式性能是最好的,但在 Kubernetes 下只能用 HostPath 挂载,不便于使用。

  • CSI NFS Pod

对于小文件的读写能力很好,适合存储代码等场景,但相较于其他方式优势并不明显。

  • Fluid NFS Pod

Fluid Runtime 提供的 NFS 性能与 Direct 直接挂载的方式性能相差不大,原因在于 Fuse Pod 的原理就是挂载 NFS 目录到当前主机目录。


微信公众号
作者
微信公众号