https://www.chenshaowen.com/en/tags/nvidia/Recent content in NVIDIA on Shaowen Chen's WebsiteHugo -- gohugo.ioen©2016 - {year}, All Rights Reserved.Tue, 09 Sep 2025 00:00:00 +0000weeklyhttps://www.chenshaowen.com/en/blog/nvidia-rtx-5090-inference-test.htmlTue, 09 Sep 2025 00:00:00 +0000Tue, 09 Sep 2025 00:00:00 +0000https://www.chenshaowen.com/en/blog/nvidia-rtx-5090-inference-test.html1. Installing the Driver Download the driver Go to https://www.nvidia.com/en-us/drivers/ and select the corresponding driver version to download. 1 wget https://us.download.nvidia.com/XFree86/Linux-x86_64/580.76.05/NVIDIA-Linux-x86_64-580.76.05.run Install the driver 1 bash NVIDIA-Linux-x86_64-580.76.05.run Check the GPUs 1 nvidia-smi 1 2 3 GPU 0: NVIDIA GeForce RTX 5090 (UUID: GPU-92fcdc58-4754-73c7-af6c-56740936817d) GPU 1: NVIDIA GeForce RTX 5090 (UUID:微信公众号AIGPUNVIDIALLMInferencePerformance TestingOperationshttps://www.chenshaowen.com/en/blog/the-use-guide-of-nvidia-dcgm.htmlWed, 13 Aug 2025 00:00:00 +0000Wed, 13 Aug 2025 00:00:00 +0000https://www.chenshaowen.com/en/blog/the-use-guide-of-nvidia-dcgm.html1. What DCGM Is DCGM (Data Center GPU Manager) is a toolset NVIDIA provides for managing and monitoring GPUs in the data center. It offers the following capabilities: GPU behavior monitoring GPU configuration management GPU policy supervision GPU health and diagnostics GPU accounting and process statistics NVSwitch configuration and monitoring微信公众号AINVIDIADCGMGPUMonitoringOperationsTroubleshootinghttps://www.chenshaowen.com/en/blog/how-to-clean-zombie-processes.htmlThu, 07 Aug 2025 00:00:00 +0000Thu, 07 Aug 2025 00:00:00 +0000https://www.chenshaowen.com/en/blog/how-to-clean-zombie-processes.html1. What a Zombie Process Is How a process is created: The parent process calls fork() to create a child process The child process calls exec() to load a new program The child process finishes executing and calls exit() or returns The parent process calls wait() or waitpid() If the parent process never calls wait() or waitpid(), the child stays in the system after it finishes, becoming a zombie process.微信公众号OperationsCleanupGPUNVIDIALinuxhttps://www.chenshaowen.com/en/blog/nvidia-environment-variable-configuration.htmlMon, 20 Jan 2025 01:00:00 +0000Mon, 20 Jan 2025 01:00:00 +0000https://www.chenshaowen.com/en/blog/nvidia-environment-variable-configuration.htmlNVIDIA_VISIBLE_DEVICES Specifies the GPU devices visible to the program 1 CUDA_VISIBLE_DEVICES=0,1 Available values: 1,2, a comma-separated list of GPU UUIDs or indices all, all GPUs none, load the driver but with no GPU access void, do not load the driver NVIDIA_DRIVER_CAPABILITIES Controls which driver libraries/binaries will be installed inside the container微信公众号AINVIDIAGPUEnvironment VariablesConfigurationhttps://www.chenshaowen.com/en/blog/nvidia-gpu-cores-and-architecture-evolution-history.htmlSun, 25 Aug 2024 00:00:00 +0000Sun, 25 Aug 2024 00:00:00 +0000https://www.chenshaowen.com/en/blog/nvidia-gpu-cores-and-architecture-evolution-history.html1. Product Lines GeForce Aimed at gamers, offering powerful graphics processing capabilities and advanced gaming technologies. Common ones include the NVIDIA GTX series, the high-end RTX series, and the Titan series. Quadro Aimed at the professional market, such as designers, engineers, scientists, and content creators. Common ones include the Quadro P series and the high-end Quadro RTX series.微信公众号AIGPUNVIDIAHardwareArchitectureHistoryhttps://www.chenshaowen.com/en/blog/use-volcano-to-run-nccl-test.htmlSun, 11 Aug 2024 00:00:00 +0000Sun, 11 Aug 2024 00:00:00 +0000https://www.chenshaowen.com/en/blog/use-volcano-to-run-nccl-test.html1. Building the nccl-test Image Check the CUDA version 1 2 3 nvidia-smi | grep "CUDA Version" | awk '{print $9}' 12.2 Write the Dockerfile 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 cat > Dockerfile << EOF FROM nvidia/cuda:12.1.0-cudnn8-devel-ubuntu22.04 ENV DEBIAN_FRONTEND=noninteractive ARG CONDA_VERSION WORKDIR /workspace ENV DEBIAN_FRONTEND=noninteractive RUN apt-get update && apt install -y openmpi-bin libopenmpi-dev ssh openssh-server net-tools vim git iputils-ping nfs-common RUN git clone https://github.微信公众号VolcanoNCCLNvidiaAITestinghttps://www.chenshaowen.com/en/blog/common-gpu-operation-and-fault-handling.htmlMon, 18 Mar 2024 00:00:00 +0000Mon, 18 Mar 2024 00:00:00 +0000https://www.chenshaowen.com/en/blog/common-gpu-operation-and-fault-handling.htmlContent referenced or recorded while handling faults, continuously updated. 1. XID Error Events XID is NVIDIA’s error code. You can retrieve it with the command: 1 dmesg -T | grep -i "NVRM: Xid" or 1 journalctl --since `date -d "10 days ago" "+%Y-%m-%d"`|grep Xid The XID lets you locate the fault.微信公众号GPUNVIDIAAIHardwareOperationsTroubleshootinghttps://www.chenshaowen.com/en/blog/speeding-up-model-inference-with-tensorrt.htmlTue, 06 Feb 2024 00:00:00 +0000Tue, 06 Feb 2024 00:00:00 +0000https://www.chenshaowen.com/en/blog/speeding-up-model-inference-with-tensorrt.html1. What Is TensorRT TensorRT is a C++ library mainly used for high-performance inference acceleration on NVIDIA GPUs. It provides both a C++ API and a Python API for integration. The mainstream deep learning frameworks TensorRT supports are: Caffe, where TensorRT can read the prototxt format directly TensorFlow, where the微信公众号AITensorRTNVIDIAGPUInferenceLLMhttps://www.chenshaowen.com/en/blog/basic-usage-of-nvidia-smi.htmlThu, 01 Feb 2024 00:00:00 +0000Thu, 01 Feb 2024 00:00:00 +0000https://www.chenshaowen.com/en/blog/basic-usage-of-nvidia-smi.html1. What Is nvidia-smi nvidia-smi stands for NVIDIA System Management Interface, the interface NVIDIA provides for managing and monitoring GPUs. nvidia-smi calls NVML. NVML stands for NVIDIA Management Library, a library that provides a set of C APIs for NVIDIA GPU monitoring and management. 1.1 Queryable State ECC error counts GPU utilization Active compute processes Clocks and PState Temperature and fan speed Power management Hardware identification 1.微信公众号AIGPUNVIDIAHardwareMonitoring ToolsOperationshttps://www.chenshaowen.com/en/blog/health-check-runs-out-of-root-run-storage.htmlTue, 23 Jan 2024 00:00:00 +0000Tue, 23 Jan 2024 00:00:00 +0000https://www.chenshaowen.com/en/blog/health-check-runs-out-of-root-run-storage.htmlIf you use nvidia-container-runtime, this is worth your attention — especially if JuiceFS is also in the picture. 1. An Alert Out of Nowhere, and I Panicked Over the weekend I was studying TensorRT LLM, and while I was at it I installed Dragonfly on the largest production cluster. Then微信公众号KubernetesNVIDIAJuiceFSExperienceTroubleshootingOperationshttps://www.chenshaowen.com/en/blog/nvidia-gpu-driver-installation.htmlThu, 28 Dec 2023 00:00:00 +0000Thu, 28 Dec 2023 00:00:00 +0000https://www.chenshaowen.com/en/blog/nvidia-gpu-driver-installation.html1. Disk Handling 1.1 Inspect Disks Inspect the new disk 1 fdisk -l Disk /dev/nvme1n1: 3.91 TiB, 4294967296000 bytes, 8388608000 sectors 1.2 Build a RAID0 If you have several small disks, a better approach is to build a RAID0 — this gives you not only a larger storage directory but also higher speed.微信公众号AIGPUNVIDIAHardwareDrivers