1. RoCE 1.1 Connectivity Requirements RDMA requires the same class of network end to end, for example the same /16 subnet. Single switch. RoCE: host NIC ── Ethernet switch (PFC) ── peer NIC Multiple switches. RoCE: host NIC ── Leaf ── Spine ── Leaf ── peer NIC When there are multiple hops, note:

1. Hardware 1.1 CPU Support for AVX/AVX-512, used for erasure coding computation and SIMD operations Support for AES-NI, used for object encryption Total storage per node Recommended vCPU count Up to 1 TB 8 Up to 10 TB 16 Up to 100 TB 32 Up to 1 PB 64 More

1. What smokeping-prober Is smokeping-prober is a tool for probing network quality. It probes network quality by sending ICMP requests to target nodes. 2. Deploying smokeping-prober 2.1 Generating the smokeping-prober.yaml Configuration File 1 2 3 4 5 6 7 8 9 10 11 cat > smokeping_prober.yaml <<EOF --- targets: - hosts: - 1.

1. Prerequisites The storage and the nodes that use it form an RDMA network The RDMA devices are configured with IP addresses If you are using Mellanox NICs, add the -with-nfsrdma parameter when installing the driver 1 ./mlnxofedinstall --with-nfsrdma 2. Starting the Server Install dependencies 1 apt install nfs-kernel-server rdma-core -y Load the kernel module 1 modprobe svcrdma Add the port to portlist 1 echo 'rdma 20049' | tee /proc/fs/nfsd/portlist Export directory 1 mkdir /data1/nfs 1 2 3 vim /etc/exports /data1/nfs *(rw,sync,no_root_squash,no_all_squash) Start the NFS service 1 systemctl start nfs-server.

Because the servers in the data center are bound to a small bandwidth, this post is mainly about using Dante to provide SOCKS5 forwarding, using some high-bandwidth servers for traffic forwarding to speed up dependency package downloads. 1. Find a High-Bandwidth Server Install speedtest-cli 1 pip3 install speedtest-cli List peer test servers 1 2 3 4 5 6 7 8 9 10 11 12 13 speedtest-cli --secure --list Retrieving speedtest.