https://www.chenshaowen.com/en/tags/gpu/Recent content in GPU on Shaowen Chen's WebsiteHugo -- gohugo.ioen©2016 - {year}, All Rights Reserved.Tue, 09 Sep 2025 00:00:00 +0000weeklyhttps://www.chenshaowen.com/en/blog/nvidia-rtx-5090-inference-test.htmlTue, 09 Sep 2025 00:00:00 +0000Tue, 09 Sep 2025 00:00:00 +0000https://www.chenshaowen.com/en/blog/nvidia-rtx-5090-inference-test.html1. Installing the Driver Download the driver Go to https://www.nvidia.com/en-us/drivers/ and select the corresponding driver version to download. 1 wget https://us.download.nvidia.com/XFree86/Linux-x86_64/580.76.05/NVIDIA-Linux-x86_64-580.76.05.run Install the driver 1 bash NVIDIA-Linux-x86_64-580.76.05.run Check the GPUs 1 nvidia-smi 1 2 3 GPU 0: NVIDIA GeForce RTX 5090 (UUID: GPU-92fcdc58-4754-73c7-af6c-56740936817d) GPU 1: NVIDIA GeForce RTX 5090 (UUID:微信公众号AIGPUNVIDIALLMInferencePerformance TestingOperationshttps://www.chenshaowen.com/en/blog/the-use-guide-of-nvidia-dcgm.htmlWed, 13 Aug 2025 00:00:00 +0000Wed, 13 Aug 2025 00:00:00 +0000https://www.chenshaowen.com/en/blog/the-use-guide-of-nvidia-dcgm.html1. What DCGM Is DCGM (Data Center GPU Manager) is a toolset NVIDIA provides for managing and monitoring GPUs in the data center. It offers the following capabilities: GPU behavior monitoring GPU configuration management GPU policy supervision GPU health and diagnostics GPU accounting and process statistics NVSwitch configuration and monitoring微信公众号AINVIDIADCGMGPUMonitoringOperationsTroubleshootinghttps://www.chenshaowen.com/en/blog/how-to-clean-zombie-processes.htmlThu, 07 Aug 2025 00:00:00 +0000Thu, 07 Aug 2025 00:00:00 +0000https://www.chenshaowen.com/en/blog/how-to-clean-zombie-processes.html1. What a Zombie Process Is How a process is created: The parent process calls fork() to create a child process The child process calls exec() to load a new program The child process finishes executing and calls exit() or returns The parent process calls wait() or waitpid() If the parent process never calls wait() or waitpid(), the child stays in the system after it finishes, becoming a zombie process.微信公众号OperationsCleanupGPUNVIDIALinuxhttps://www.chenshaowen.com/en/blog/kubernetes-gpu-virtualization.htmlSat, 14 Jun 2025 00:00:00 +0000Sat, 14 Jun 2025 00:00:00 +0000https://www.chenshaowen.com/en/blog/kubernetes-gpu-virtualization.html1. k8s-device-plugin https://github.com/NVIDIA/k8s-device-plugin is the official Kubernetes device plugin from NVIDIA, used to manage and allocate NVIDIA GPU resources in a Kubernetes cluster. By interacting with kubelet, k8s-device-plugin automatically discovers and registers GPU devices and exposes them to the Kubernetes scheduler as resources. It supports many GPU models and is able to handle GPU partitioning and sharing.微信公众号KubernetesGPUAIVirtualizationhttps://www.chenshaowen.com/en/blog/nvidia-environment-variable-configuration.htmlMon, 20 Jan 2025 01:00:00 +0000Mon, 20 Jan 2025 01:00:00 +0000https://www.chenshaowen.com/en/blog/nvidia-environment-variable-configuration.htmlNVIDIA_VISIBLE_DEVICES Specifies the GPU devices visible to the program 1 CUDA_VISIBLE_DEVICES=0,1 Available values: 1,2, a comma-separated list of GPU UUIDs or indices all, all GPUs none, load the driver but with no GPU access void, do not load the driver NVIDIA_DRIVER_CAPABILITIES Controls which driver libraries/binaries will be installed inside the container微信公众号AINVIDIAGPUEnvironment VariablesConfigurationhttps://www.chenshaowen.com/en/blog/sr-iov-technique.htmlSun, 03 Nov 2024 00:00:00 +0000Sun, 03 Nov 2024 00:00:00 +0000https://www.chenshaowen.com/en/blog/sr-iov-technique.html1. What SR-IOV Technology Is SR-IOV (Single Root I/O Virtualization) is a virtualization technology that lets virtual machines and containers access physical hardware resources directly, improving I/O performance and reducing host CPU consumption. As shown in the figure above, SR-IOV partitions a single physical device (for example a network interface微信公众号VirtualizationSR-IOVNetworkOperationsGPUhttps://www.chenshaowen.com/en/blog/how-to-enable-gds-on-gpu-host.htmlWed, 11 Sep 2024 00:00:00 +0000Wed, 11 Sep 2024 00:00:00 +0000https://www.chenshaowen.com/en/blog/how-to-enable-gds-on-gpu-host.html1. What GDS (GPUDirectStorage) Is GDS lets an RDMA NIC access GPU memory directly, which helps increase the IO bandwidth for GPU applications reading and writing files, reduces IO latency, and lowers their CPU load. Once the client has GDS enabled, files are opened in O_DIRECT mode and the client no longer caches file data.微信公众号GDSGPUAIOperationsStoragehttps://www.chenshaowen.com/en/blog/nvidia-gpu-cores-and-architecture-evolution-history.htmlSun, 25 Aug 2024 00:00:00 +0000Sun, 25 Aug 2024 00:00:00 +0000https://www.chenshaowen.com/en/blog/nvidia-gpu-cores-and-architecture-evolution-history.html1. Product Lines GeForce Aimed at gamers, offering powerful graphics processing capabilities and advanced gaming technologies. Common ones include the NVIDIA GTX series, the high-end RTX series, and the Titan series. Quadro Aimed at the professional market, such as designers, engineers, scientists, and content creators. Common ones include the Quadro P series and the high-end Quadro RTX series.微信公众号AIGPUNVIDIAHardwareArchitectureHistoryhttps://www.chenshaowen.com/en/blog/common-ai-base-images-and-run-command.htmlThu, 28 Mar 2024 00:01:00 +0000Thu, 28 Mar 2024 00:01:00 +0000https://www.chenshaowen.com/en/blog/common-ai-base-images-and-run-command.html1. What the Image Tag Suffixes Mean base/cuda: includes the CUDA runtime runtime: builds on base, adding the CUDA math libraries and the NCCL and cuDNN runtimes devel: builds on runtime, adding headers and the development tools used to build CUDA images; particularly useful for multi-stage builds cuddn: builds on the above, adding the cuDNN neural network acceleration library微信公众号AIImagesContainersDockerGPULLMOperationsR&Dhttps://www.chenshaowen.com/en/blog/common-gpu-operation-and-fault-handling.htmlMon, 18 Mar 2024 00:00:00 +0000Mon, 18 Mar 2024 00:00:00 +0000https://www.chenshaowen.com/en/blog/common-gpu-operation-and-fault-handling.htmlContent referenced or recorded while handling faults, continuously updated. 1. XID Error Events XID is NVIDIA’s error code. You can retrieve it with the command: 1 dmesg -T | grep -i "NVRM: Xid" or 1 journalctl --since `date -d "10 days ago" "+%Y-%m-%d"`|grep Xid The XID lets you locate the fault.微信公众号GPUNVIDIAAIHardwareOperationsTroubleshootinghttps://www.chenshaowen.com/en/blog/speeding-up-model-inference-with-tensorrt.htmlTue, 06 Feb 2024 00:00:00 +0000Tue, 06 Feb 2024 00:00:00 +0000https://www.chenshaowen.com/en/blog/speeding-up-model-inference-with-tensorrt.html1. What Is TensorRT TensorRT is a C++ library mainly used for high-performance inference acceleration on NVIDIA GPUs. It provides both a C++ API and a Python API for integration. The mainstream deep learning frameworks TensorRT supports are: Caffe, where TensorRT can read the prototxt format directly TensorFlow, where the微信公众号AITensorRTNVIDIAGPUInferenceLLMhttps://www.chenshaowen.com/en/blog/using-triton-server-and-tensorrt-llm-under-container.htmlSat, 03 Feb 2024 08:05:48 +0000Sat, 03 Feb 2024 08:05:48 +0000https://www.chenshaowen.com/en/blog/using-triton-server-and-tensorrt-llm-under-container.html1. Compiling Models with TensorRT-LLM 1.1 Introduction to TensorRT-LLM When using TensorRT, you usually need to convert the model to ONNX format, then convert the ONNX to TensorRT format, and finally run inference in TensorRT or Triton Server. But this conversion process is not simple, and you often run into微信公众号AITritonTensorRTLLMInferenceGPUOperationsContainersLearninghttps://www.chenshaowen.com/en/blog/basic-usage-of-nvidia-smi.htmlThu, 01 Feb 2024 00:00:00 +0000Thu, 01 Feb 2024 00:00:00 +0000https://www.chenshaowen.com/en/blog/basic-usage-of-nvidia-smi.html1. What Is nvidia-smi nvidia-smi stands for NVIDIA System Management Interface, the interface NVIDIA provides for managing and monitoring GPUs. nvidia-smi calls NVML. NVML stands for NVIDIA Management Library, a library that provides a set of C APIs for NVIDIA GPU monitoring and management. 1.1 Queryable State ECC error counts GPU utilization Active compute processes Clocks and PState Temperature and fan speed Power management Hardware identification 1.微信公众号AIGPUNVIDIAHardwareMonitoring ToolsOperationshttps://www.chenshaowen.com/en/blog/nvidia-gpu-driver-installation.htmlThu, 28 Dec 2023 00:00:00 +0000Thu, 28 Dec 2023 00:00:00 +0000https://www.chenshaowen.com/en/blog/nvidia-gpu-driver-installation.html1. Disk Handling 1.1 Inspect Disks Inspect the new disk 1 fdisk -l Disk /dev/nvme1n1: 3.91 TiB, 4294967296000 bytes, 8388608000 sectors 1.2 Build a RAID0 If you have several small disks, a better approach is to build a RAID0 — this gives you not only a larger storage directory but also higher speed.微信公众号AIGPUNVIDIAHardwareDrivershttps://www.chenshaowen.com/en/blog/how-to-install-gpu-driver-in-centos.htmlSat, 15 Feb 2020 00:00:00 +0000Sat, 15 Feb 2020 00:00:00 +0000https://www.chenshaowen.com/en/blog/how-to-install-gpu-driver-in-centos.htmlUsing CentOS 7.7 and a Tesla P100 GPU as an example. 1. Preparing the Base Environment Install the lspci command 1 yum install -y pciutils Check whether the GPU supports CUDA 1 2 3 lspci | grep -i nvidia 00:09.0 3D controller: NVIDIA Corporation GP100GL [Tesla P100 PCIe 12GB] (rev a1) List of CUDA-capable GPUs: https://developer.微信公众号DockerGPUCentOSOperationsDrivers