AI
Fluid Directly Mounting S3 and Performance Testing
· β˜• 6 min read
1. Packaging the Fluid Runtime Image Create the fluid_config_init.py script 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 #!/usr/bin/env python import json rawStr = "" with open("/etc/fluid/config.json", "r") as f: rawStr = f.

Fluid Using Lustre Runtime and Performance Testing
· β˜• 8 min read
1. Analyzing How Fluid Mounts NFS Storage Inspect the Fuse Pod 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 kubectl get pod nfs-demo-fuse-f9wg8 -oyaml apiVersion: v1 kind: Pod metadata: generateName: nfs-demo-fuse- spec: containers: - command: - /usr/local/bin/entrypoint.

Fluid Using NFS Runtime and Performance Testing
· β˜• 5 min read
1. Creating a Dataset 1 2 3 4 5 6 7 8 9 10 kubectl apply -f - <<EOF apiVersion: data.fluid.io/v1alpha1 kind: Dataset metadata: name: nfs-demo spec: mounts: - mountPoint: x.x.x.x:/x-x/ name: nfs-demo EOF 2. Creating a Runtime 1 2 3 4 5 6 7 8 9 10 11 12 13 14 kubectl apply -f - <<EOF apiVersion: data.

Aligning with Ops: Rewriting Ops Copilot with a New Approach, Updated
· β˜• 5 min read
1. Making Ops Copilot an Ops Coilot In September 2023, I wrote a version of Ops Copilot, and published an article about it, I’m Writing a Copilot for My Ops Tool. The effect it achieved was like this: 1 2 3 4 5 6 7 8 9 10 Opscli> 打

Model Parallel Training Techniques
· β˜• 4 min read
1. Data Parallelism Training steps: The master device loads the model and copies the model parameters to every worker device The master device splits the training data along the batch dimension and passes each batch to every worker device Each worker device performs training The master device aggregates the gradients from each worker device and updates the model parameters The master device broadcasts the model parameters to every worker device, preparing for the next batch of training Core idea: