1. Why Migrate Metadata Storage from Redis to PGSQL PGSQL is cheaper Redis stores metadata in memory, while PGSQL stores metadata on disk, so the cost difference is obvious. PGSQL performance is tunable Different PGSQL services offer different performance IOPS. If you do not have a continuously high performance requirement, PGSQL is a good choice.

1. Building the hccl-test Image Download the dependency packages Python-3.8.18.tgz Ascend-cann-toolkit_8.0.RC2_linux-x86_64.run Ascend-cann-kernels-910b_8.0.RC2_linux.run mpich-3.2.1.tar.gz If downloading is inconvenient, you can also copy them straight out of the image I built. Write the Dockerfile 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 FROM ubuntu:22.

1. Building the nccl-test Image Check the CUDA version 1 2 3 nvidia-smi | grep "CUDA Version" | awk '{print $9}' 12.2 Write the Dockerfile 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 cat > Dockerfile << EOF FROM nvidia/cuda:12.1.0-cudnn8-devel-ubuntu22.04 ENV DEBIAN_FRONTEND=noninteractive ARG CONDA_VERSION WORKDIR /workspace ENV DEBIAN_FRONTEND=noninteractive RUN apt-get update && apt install -y openmpi-bin libopenmpi-dev ssh openssh-server net-tools vim git iputils-ping nfs-common RUN git clone https://github.

1. top Shows Node Resource Usage Above 100% 1 2 3 4 5 6 kubectl top node NAME CPU(cores) CPU% MEMORY(bytes) MEMORY% master-1 995m 16% 13760Mi 118% master-2 827m 13% 10672Mi 92% master-3 889m 14% 10244Mi 88% This is because usage is computed against allocatable resources by default, which excludes the portion reserved by Kubelet.

1. Checking the CPU Check the CPU model 1 lscpu Check the instruction set 1 cat /proc/cpuinfo | grep -iE "flags|instruction set" 2. Checking Memory Memory usage and size 1 free -h Memory module model 1 dmidecode -t memory 3. Checking Disks Check disk mounts 1 lsblk -o NAME,TYPE,SIZE,MODEL,UUID,MOUNTPOINT Check disk usage 1 df -H | grep -vE '^Filesystem|tmpfs|cdrom|loop|udev' | awk '{ print $5 "/" $2 " " $1 }' |grep " "/ Check disk speed 1 hdparm -t /dev/sda3 Check disk temperature and more 1 apt install smartmontools 1 (for disk in /dev/sd?