1. What the Image Tag Suffixes Mean base/cuda: includes the CUDA runtime runtime: builds on base, adding the CUDA math libraries and the NCCL and cuDNN runtimes devel: builds on runtime, adding headers and the development tools used to build CUDA images; particularly useful for multi-stage builds cuddn: builds on the above, adding the cuDNN neural network acceleration library

1. Compiling Models with TensorRT-LLM 1.1 Introduction to TensorRT-LLM When using TensorRT, you usually need to convert the model to ONNX format, then convert the ONNX to TensorRT format, and finally run inference in TensorRT or Triton Server. But this conversion process is not simple, and you often run into

Unlike on CentOS or Ubuntu, where we feel that mv is faster than cp, when you build an image with a Dockerfile, using Run cp is faster than Run mv. This post presents the test and verification data behind that. 1. Test Setup Machine environment Ubuntu 20.04.1 LTS 32C

No. 1. Background When building a reliable, stable operations system on top of Kubernetes, destroying and creating virtual machines (VMs) is the norm. VMs provide compute and memory resources, while external storage is exposed to PVCs in the cluster through a StorageClass. Against this backdrop, how to initialize a VM quickly becomes a new challenge.

A Docker mirror can only accelerate images from docker.io, not images from private registries. 1. Why You Need a Private Registry Mirror Rate limiting on the public network Docker Hub pull rate limits Reduce image pull time 2. Create a Registry Image Acceleration Service Generate a configuration file version: 0.