1. Compiling Models with TensorRT-LLM 1.1 Introduction to TensorRT-LLM When using TensorRT, you usually need to convert the model to ONNX format, then convert the ONNX to TensorRT format, and finally run inference in TensorRT or Triton Server. But this conversion process is not simple, and you often run into

1. What Is nvidia-smi nvidia-smi stands for NVIDIA System Management Interface, the interface NVIDIA provides for managing and monitoring GPUs. nvidia-smi calls NVML. NVML stands for NVIDIA Management Library, a library that provides a set of C APIs for NVIDIA GPU monitoring and management. 1.1 Queryable State ECC error counts GPU utilization Active compute processes Clocks and PState Temperature and fan speed Power management Hardware identification 1.

1. Introduction to Fluid Below is the Fluid architecture diagram from https://github.com/fluid-cloudnative/fluid: Fluid abstracts two concepts: Dataset, a collection of data, the abstraction from the user’s point of view Runtime, the abstraction of the real services such as data storage and acceleration Fluid mainly solves the problems of using traditional caching systems on Kubernetes:

Note that --block-size 4096 in this post means 4GB; using --block-size 4 would be more reasonable and would give better write performance. 1. Environment Preparation Enter a safe directory 1 mkdir -p /data/test && cd /data/test All the test tasks are carried out in this directory. Create a separate directory for Redis 1 mkdir redis-data It is recommended to create a new directory, because Redis will change the Owner of the files in the current directory to systemd-coredump.

If you use nvidia-container-runtime, this is worth your attention — especially if JuiceFS is also in the picture. 1. An Alert Out of Nowhere, and I Panicked Over the weekend I was studying TensorRT LLM, and while I was at it I installed Dragonfly on the largest production cluster. Then