In a PD disaggregated deployment, heterogeneous GPU models are often used to deploy the model across machines, which multiplies the cross-machine communication pressure. RDMA devices are usually brought in to accelerate kvcache transfer between nodes so as to achieve a lower FTTL. This post describes how to test eRDMA devices

1. Why Operations Events Less invasive Operations teams usually already have observations accumulated in the Metrics and Log dimensions. Operations events can be a new entry point: introduce new operational capabilities without affecting the stability of existing systems. Better suited to automation Metrics represent system state, Log represents specific code behavior, while Event represents changes in components.

1. Prompt Chaining Prompt chaining breaks a task down into a series of steps, where each LLM call processes the output of the previous step. 2. Routing Routing classifies the input and directs it to a specialized follow-up task. 3. Parallelization An LLM can sometimes work on a task concurrently and aggregate its outputs programmatically.

1. Handling Deterministic Faults For faults that have a specific handling procedure, just let the Agent handle them and send a notification to spread the word. For similar automated handling, we have exception load deletion at the application layer, disk cleanup at the node layer, masking an offline GPU card, masking a node, and so on.

1. Single Disk 1.1 Creating the File System Wipe the disk 1 wipefs -a /dev/nvme0n1 Format the disk 1 mkfs.xfs /dev/nvme0n1 Mount the disk 1 2 mkdir -p /mnt/nvme0n1-test mount /dev/nvme0n1 /mnt/nvme0n1-test 1.2 Performance Test 4k random write 1 2 3 4 echo 3 > /proc/sys/vm/drop_caches fio --name=4k_randwrite --rw=randwrite --bs=4k --ioengine=libaio --direct=1 \ --numjobs=8 --iodepth=32 --size=20G --time_based --runtime=120 --group_reporting \ --allow_mounted_write=1 --filename=/mnt/nvme0n1-test/fio-4k-randwrite.