1. 创建 Dataset
1
2
3
4
5
6
7
8
9
10
| kubectl apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
name: my-s3
type: Opaque
stringData:
aws.accessKeyId: xxx
aws.secretKey: xxx
EOF
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
| kubectl apply -f - <<EOF
apiVersion: data.fluid.io/v1alpha1
kind: Dataset
metadata:
name: my-s3
spec:
mounts:
- mountPoint: s3://BUCKET/
name: s3
options:
alluxio.underfs.s3.endpoint: ks3-cn-beijing-internal.ksyun.com
alluxio.underfs.s3.disable.dns.buckets: "false"
encryptOptions:
- name: aws.accessKeyId
valueFrom:
secretKeyRef:
name: my-s3
key: aws.accessKeyId
- name: aws.secretKey
valueFrom:
secretKeyRef:
name: my-s3
key: aws.secretKey
accessModes:
- ReadWriteMany
EOF
|
2. 创建 Runtime
1
2
3
4
5
6
7
8
9
10
11
12
13
14
| kubectl apply -f - <<EOF
apiVersion: data.fluid.io/v1alpha1
kind: AlluxioRuntime
metadata:
name: my-s3
spec:
tieredstore:
levels:
- mediumtype: MEM
path: /dev/shm
quota: 50Gi
high: "0.95"
low: "0.7"
EOF
|
3. 创建测试 Pod
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
| kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
name: s3-demo
spec:
containers:
- name: demo
image: shaowenchen/demo-ubuntu
volumeMounts:
- mountPath: /data
name: data
volumes:
- name: data
persistentVolumeClaim:
claimName: my-s3
EOF
|
4. 性能测试
1
2
3
4
5
6
7
8
9
| time dd if=/dev/zero of=./dd.txt bs=4M count=250
250+0 records in
250+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 57.8283 s, 18.1 MB/s
real 1m14.210s
user 0m0.000s
sys 0m0.363s
|
1
2
3
4
5
| time cp ./dd.txt /dev/null
real 0m13.823s
user 0m0.000s
sys 0m0.386s
|
1
2
3
4
5
| time cp ./dd.txt /dev/null
real 0m0.226s
user 0m0.004s
sys 0m0.221s
|
5. 清理
1
2
3
4
| kubectl delete pod s3-demo
kubectl delete alluxioruntime my-s3
kubectl delete dataset my-s3
kubectl delete secret my-s3
|
6. 总结
使用 Fluid 直接将 S3 挂载为 PVC 性能不算好，以上的测试桶和集群在一个区域。
- 已经挂载的 PVC 并不能看到 S3 桶的更新。也就是说，PVC 只是 S3 挂载瞬间的 Snapshort
- 不要在 PVC 中创建目录，目录不会同步到 S3 桶
- 不要在 PVC 中创建文件，文件不会同步到 S3 桶
- 读写速度都很慢
使用 s3fs + ThinRuntimeProfile 的方式应该更好，但目前暂时没有业务需求，另外如果是 OSS 可以使用 JindoRuntime 支持对象存储的加速。
