https://www.chenshaowen.com/en/tags/images/Recent content in Images on Shaowen Chen's WebsiteHugo -- gohugo.ioen©2016 - {year}, All Rights Reserved.Mon, 28 Jul 2025 00:00:00 +0000weeklyhttps://www.chenshaowen.com/en/blog/deploy-dragonfly-on-kubernetes.htmlMon, 28 Jul 2025 00:00:00 +0000Mon, 28 Jul 2025 00:00:00 +0000https://www.chenshaowen.com/en/blog/deploy-dragonfly-on-kubernetes.html1. Deploy NFS Storage 1.1 Configure the NFS Service on the Storage Node Install the NFS service on the storage node 1 apt-get install -y nfs-kernel-server Start the NFS service and enable it at boot 1 systemctl start nfs-server.service & systemctl enable nfs-server.service Configure the NFS shared directory 1 mkdir -p /data/nfs 1 vim /etc/exports /data/nfs *(rw,sync,no_root_squash,no_all_squash) Restart the NFS service 1 systemctl restart nfs-server.微信公众号KubernetesDragonflyImagesImage AccelerationAIOperationshttps://www.chenshaowen.com/en/blog/use-nerdctl-to-build-multi-architecture-images.htmlFri, 17 Jan 2025 00:00:00 +0000Fri, 17 Jan 2025 00:00:00 +0000https://www.chenshaowen.com/en/blog/use-nerdctl-to-build-multi-architecture-images.html1. Installing Nerdctl Install Opscli 1 curl -sfL https://raw.githubusercontent.com/shaowenchen/ops/main/getcli.sh |VERSION=latest sh - Install Nerdctl 1 opscli task -f install-nerdctl --arch amd64 2. BuildKit Download Buildkit 1 wget https://github.com/moby/buildkit/releases/download/v0.19.0-rc2/buildkit-v0.19.0-rc2.linux-amd64.tar.gz Install Buildkit 1 2 tar xvf buildkit-*.tar.gz mv bin/* /usr/local/bin/ Configure Buildkitd 1 mkdir -p /etc/buildkit /data/buildkit 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 cat > /etc/buildkit/buildkitd.微信公众号Multi-ArchImagesCINerdctlKubernetesOperationsLearningCICDcontainerdBuildhttps://www.chenshaowen.com/en/blog/common-ai-base-images-and-run-command.htmlThu, 28 Mar 2024 00:01:00 +0000Thu, 28 Mar 2024 00:01:00 +0000https://www.chenshaowen.com/en/blog/common-ai-base-images-and-run-command.html1. What the Image Tag Suffixes Mean base/cuda: includes the CUDA runtime runtime: builds on base, adding the CUDA math libraries and the NCCL and cuDNN runtimes devel: builds on runtime, adding headers and the development tools used to build CUDA images; particularly useful for multi-stage builds cuddn: builds on the above, adding the cuDNN neural network acceleration library微信公众号AIImagesContainersDockerGPULLMOperationsR&Dhttps://www.chenshaowen.com/en/blog/distributing-image-with-dragonfly-v2.htmlSat, 13 Jan 2024 11:22:55 +0000Sat, 13 Jan 2024 11:22:55 +0000https://www.chenshaowen.com/en/blog/distributing-image-with-dragonfly-v2.html1. Introduction to Dragonfly The Dragonfly documentation at https://d7y.io/zh/docs/ already covers things in detail. Here is just a brief introduction to the main components of V2: Manager, which provides the UI, user management, cluster monitoring, task management, and other features Scheduler, which schedules traffic between Peers and provides preheating and other features Seed Peer, the back-to-source node used to download data from origin sites (Harbor, Docker.微信公众号AIDragonflyKubernetesImagesImageNydusConfigurationPracticeOperationshttps://www.chenshaowen.com/en/blog/nydus-configuration-and-practice.htmlSat, 06 Jan 2024 11:22:55 +0000Sat, 06 Jan 2024 11:22:55 +0000https://www.chenshaowen.com/en/blog/nydus-configuration-and-practice.htmlStatistics show that most files inside a container are never used. Based on this characteristic, Nydus defines a custom filesystem in the Rafs format, enabling on-demand loading of image files to solve the slow startup and storage consumption caused by large images. In AI scenarios, whether for inference or training, images often start at several GB, or even tens of GB, so Nydus is very well suited.微信公众号AINydusDockerImagesConfigurationPracticeOperationshttps://www.chenshaowen.com/en/blog/troubleshoot-slow-io-when-building-dockerfile.htmlThu, 02 Feb 2023 00:00:00 +0000Thu, 02 Feb 2023 00:00:00 +0000https://www.chenshaowen.com/en/blog/troubleshoot-slow-io-when-building-dockerfile.html1. The Problem Project details: File size 5.6 GB File count 529352 Dockerfile 1 2 3 FROM golang:1.13 COPY ./ /go/src/code The build command and its output are as follows: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 time DOCKER_BUILDKIT=1 docker build --no-cache -t test:v3 -f Dockerfile .微信公众号DockerImagesCICDBuildTroubleshootingOperationshttps://www.chenshaowen.com/en/blog/run-mv-is-slower-than-cp-in-dockerfile.htmlMon, 30 Jan 2023 00:00:00 +0000Mon, 30 Jan 2023 00:00:00 +0000https://www.chenshaowen.com/en/blog/run-mv-is-slower-than-cp-in-dockerfile.htmlUnlike on CentOS or Ubuntu, where we feel that mv is faster than cp, when you build an image with a Dockerfile, using Run cp is faster than Run mv. This post presents the test and verification data behind that. 1. Test Setup Machine environment Ubuntu 20.04.1 LTS 32C微信公众号DockerContainersCICDImagesOperationshttps://www.chenshaowen.com/en/blog/hijack-docker-io-req-to-private-repository.htmlMon, 18 Jul 2022 00:00:00 +0000Mon, 18 Jul 2022 00:00:00 +0000https://www.chenshaowen.com/en/blog/hijack-docker-io-req-to-private-repository.html1. Self-signing a *.docker.io Domain Certificate 1.1 Creating a CA Certificate Generate the CA certificate private key 1 openssl genrsa -out ca.key 4096 Generate the CA certificate 1 2 3 4 openssl req -x509 -new -nodes -sha512 -days 3650 \ -subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=chenshaowen.com" \ -key ca.key \ -out ca.crt 1.2 Creating a *.微信公众号JenkinsKubernetesDockerImagesOperationsSecurityhttps://www.chenshaowen.com/en/blog/a-image-management-and-distribution-case-based-on-harbor-and-registry.htmlSat, 16 Jul 2022 00:00:00 +0000Sat, 16 Jul 2022 00:00:00 +0000https://www.chenshaowen.com/en/blog/a-image-management-and-distribution-case-based-on-harbor-and-registry.html1. The Challenge of a Cross-Region Harbor If all you need is to store image data in a simple way, Registry is an excellent choice as an image repository. Registry not only supports multiple storage backends, but can also be configured with HTTPS certificates and access credentials. Worth noting: Harbor微信公众号HarborRegistryImagesDistributionKubernetesOperationsLearningMulti-RegionAccelerationCICDhttps://www.chenshaowen.com/en/blog/a-global-images-distribution-network.htmlSun, 27 Mar 2022 14:00:00 +0000Sun, 27 Mar 2022 14:00:00 +0000https://www.chenshaowen.com/en/blog/a-global-images-distribution-network.html1. Global Network Planning Many global, multi-region infrastructures do not put much thought into network planning at the outset. Once the business grows complex enough, they are forced into network adjustments and optimization. Yet any major network change has a huge impact on the business. In the end you are stuck in a dilemma, able to do nothing but invest more people, carry the baggage of history, and walk along the edge of a cliff again and again.微信公众号ImagesHarborNetworkKubernetesOperationsDistributionMulti-RegionLearninghttps://www.chenshaowen.com/en/blog/how-to-run-a-private-registry-mirror.htmlThu, 29 Apr 2021 00:00:00 +0000Thu, 29 Apr 2021 00:00:00 +0000https://www.chenshaowen.com/en/blog/how-to-run-a-private-registry-mirror.htmlA Docker mirror can only accelerate images from docker.io, not images from private registries. 1. Why You Need a Private Registry Mirror Rate limiting on the public network Docker Hub pull rate limits Reduce image pull time 2. Create a Registry Image Acceleration Service Generate a configuration file version: 0.微信公众号DockerContainersImagesOperationsRegistryAccelerationhttps://www.chenshaowen.com/en/blog/a-sample-intro-to-cloud-native-buildpacks.htmlFri, 05 Feb 2021 00:00:00 +0000Fri, 05 Feb 2021 00:00:00 +0000https://www.chenshaowen.com/en/blog/a-sample-intro-to-cloud-native-buildpacks.html1. Buildpacks, an Old Tree Blooms Anew The Buildpacks project was first started by Heroku in 2011 and was widely adopted by PaaS platforms such as Cloud Foundry. In an earlier document, Buildpack for PaaS Deployment, I demonstrated how to deploy a Django application to Heroku. The shortcoming of Buildpacks was that the artifact it produced was in Droplet format, which could not be adapted directly to container platforms.微信公众号DevOpsBuildpackContainersImageshttps://www.chenshaowen.com/en/blog/the-daemon-less-tools-of-kaniko.htmlFri, 11 Dec 2020 08:00:00 +0000Fri, 11 Dec 2020 08:00:00 +0000https://www.chenshaowen.com/en/blog/the-daemon-less-tools-of-kaniko.html1. daemon-less Image Build Tools 1.1 What Is a daemon-less Image Build Tool In a CICD pipeline, image builds come up often, and the conventional approach is to build with Docker in Docker or Docker out of Docker. For details, see: How to Use Docker in Docker In fact, to avoid monopolies and advance the industry, a unified OCI image format specification based on the Docker image format was defined long ago.微信公众号KanikoDevOpsImagesCICDKubernetesOperationshttps://www.chenshaowen.com/en/blog/how-to-cross-the-limit-of-dockerhub.htmlWed, 09 Dec 2020 00:00:00 +0000Wed, 09 Dec 2020 00:00:00 +0000https://www.chenshaowen.com/en/blog/how-to-cross-the-limit-of-dockerhub.html1. DockerHub Limits In the end, there is no getting around this error: 1 Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit Starting November 2, 2020, DockerHub officially began limiting the pull rate for non-paying users:微信公众号DockerImagesDockerHubOperationsTroubleshootingRate LimitingProxyhttps://www.chenshaowen.com/en/blog/deploy-static-server-by-docker-image.htmlWed, 16 Sep 2020 00:00:00 +0000Wed, 16 Sep 2020 00:00:00 +0000https://www.chenshaowen.com/en/blog/deploy-static-server-by-docker-image.html1. Why Deploy Independently with an Image Better SEO With GitHub Pages, deploying a static site is very convenient. Configure it once, and every commit deploys and updates automatically. Add jsDelivr and Cloudflare acceleration on top, and among free options it is already production-ready. Unfortunately, Baidu’s search engine indexes GitHub Pages sites very slowly, or not at all.微信公众号HexoDockerImagesBlogOperationshttps://www.chenshaowen.com/en/blog/docker-image-under-multi-arch.htmlSun, 05 Jul 2020 00:00:00 +0000Sun, 05 Jul 2020 00:00:00 +0000https://www.chenshaowen.com/en/blog/docker-image-under-multi-arch.html1. Enabling Docker’s experimental Feature First, enable Docker’s experimental feature, so the commands below are available. Edit vim ~/.docker/config.json and add the following: 1 2 3 4 { "experimental": "enabled", "debug": true } Note that this is not the /etc/docker/daemon.json file, and Docker does not need to be restarted. 2. Docker Images Starting with Docker 1.微信公众号DockerImagesMulti-ArchmanifestOperationshttps://www.chenshaowen.com/en/blog/support-https-access-harbor-using-self-signed-cert.htmlSat, 18 Apr 2020 00:00:00 +0000Sat, 18 Apr 2020 00:00:00 +0000https://www.chenshaowen.com/en/blog/support-https-access-harbor-using-self-signed-cert.htmlIn an earlier post, Installing Harbor with Helm, I described in detail how to install Ingress and Harbor and finally push an image successfully. There the domain was publicly reachable and the certificate was issued by a certificate authority. In an intranet environment, however, we need to use an internal微信公众号HarborKubernetesHttpsImagesCertificatesOperationshttps://www.chenshaowen.com/en/blog/an-enterprise-class-registry-of-harbor.htmlThu, 12 Sep 2019 00:00:00 +0000Thu, 12 Sep 2019 00:00:00 +0000https://www.chenshaowen.com/en/blog/an-enterprise-class-registry-of-harbor.html1. Requirements for Building Harbor Harbor hardware requirements: CPU, at least 2 cores, 4 cores is better Mem, at least 4 GB, 8 GB is better Disk, at least 40 GB, 160 GB is better Docker version requirements: 17.06.0 or later To build Harbor on Kubernetes, you can refer to the documentation, Building Harbor with Helm.微信公众号HarborImagesContainersKubernetesOperationshttps://www.chenshaowen.com/en/blog/using-s2i-to-build-cloud-native-applications.htmlMon, 26 Aug 2019 00:00:00 +0000Mon, 26 Aug 2019 00:00:00 +0000https://www.chenshaowen.com/en/blog/using-s2i-to-build-cloud-native-applications.html1. What Problem S2I Solves It is foreseeable that a large number of applications will be deployed as containers in the future. Container platforms care about images and containers, while application development cares about business code, and ultimately the code needs to be deployed as an image. From code to image — that is Source To Image, or S2I.微信公众号S2IDockerCloud NativeImagesOperationsR&D