https://www.chenshaowen.com/en/tags/cicd/Recent content in CICD on Shaowen Chen's WebsiteHugo -- gohugo.ioen©2016 - {year}, All Rights Reserved.Fri, 17 Jan 2025 00:00:00 +0000weeklyhttps://www.chenshaowen.com/en/blog/use-nerdctl-to-build-multi-architecture-images.htmlFri, 17 Jan 2025 00:00:00 +0000Fri, 17 Jan 2025 00:00:00 +0000https://www.chenshaowen.com/en/blog/use-nerdctl-to-build-multi-architecture-images.html1. Installing Nerdctl Install Opscli 1 curl -sfL https://raw.githubusercontent.com/shaowenchen/ops/main/getcli.sh |VERSION=latest sh - Install Nerdctl 1 opscli task -f install-nerdctl --arch amd64 2. BuildKit Download Buildkit 1 wget https://github.com/moby/buildkit/releases/download/v0.19.0-rc2/buildkit-v0.19.0-rc2.linux-amd64.tar.gz Install Buildkit 1 2 tar xvf buildkit-*.tar.gz mv bin/* /usr/local/bin/ Configure Buildkitd 1 mkdir -p /etc/buildkit /data/buildkit 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 cat > /etc/buildkit/buildkitd.微信公众号Multi-ArchImagesCINerdctlKubernetesOperationsLearningCICDcontainerdBuildhttps://www.chenshaowen.com/en/blog/event-driven-workflow-using-argo-events.htmlWed, 27 Mar 2024 00:00:00 +0000Wed, 27 Mar 2024 00:00:00 +0000https://www.chenshaowen.com/en/blog/event-driven-workflow-using-argo-events.html1. How Argo Events Works The diagram above is from the official Argo Events website. An event processing system has three important parts: Ingesting event sources, which corresponds to the Event Source Distributing events, which corresponds to the Event Sensor Consuming events, which corresponds to the Event Trigger The event messages are stored in the EventBus, which uses NATS by default.微信公众号ArgoArgo EventsKubernetesCICDEvent-DrivenOperationshttps://www.chenshaowen.com/en/blog/introduction-to-argo-core-components.htmlFri, 23 Feb 2024 00:00:00 +0000Fri, 23 Feb 2024 00:00:00 +0000https://www.chenshaowen.com/en/blog/introduction-to-argo-core-components.html1. Introduction to Argo Argo is an open source project that provides a set of tools on Kubernetes for building and running applications. The core components of Argo mainly include the following: Argo CD Argo CD is a GitOps-based continuous delivery tool that allows users to declaratively manage the deployment of applications in a Kubernetes cluster.微信公众号ArgoArgo CDKubernetesCICDDevOpsOperationshttps://www.chenshaowen.com/en/blog/monitor-service-accessibility-with-upptime-for-free.htmlSat, 12 Aug 2023 00:00:00 +0000Sat, 12 Aug 2023 00:00:00 +0000https://www.chenshaowen.com/en/blog/monitor-service-accessibility-with-upptime-for-free.html1. What Needs Dial Testing GPT was all the rage this year, and I deployed and developed several applications and mini programs to learn from. Of course, following the principle of helping vendors test their features, right now only the GPT 3.5 API costs a little every day — the微信公众号GitHubActionsMonitoringDial TestingOperationsLearningAvailabilityFreeCICDBest Practiceshttps://www.chenshaowen.com/en/blog/the-scope-of-credential-in-building.htmlThu, 06 Jul 2023 00:00:00 +0000Thu, 06 Jul 2023 00:00:00 +0000https://www.chenshaowen.com/en/blog/the-scope-of-credential-in-building.htmlIn the client, we have already seen the Docker CLI passing credentials to the Docker Daemon via X-Registry-Config when sending the build context. But recent build feedback still shows some phenomena that cannot be explained. This post mainly runs some basic tests to make troubleshooting easier. 1. Building under host Docker The Docker Daemon is started with root privileges.微信公众号CICDDevOpsDockerCredentialsKubernetesOperationsTroubleshootinghttps://www.chenshaowen.com/en/blog/custom-cluster-scheduler-to-optimize-tekton.htmlWed, 26 Apr 2023 10:00:00 +0000Wed, 26 Apr 2023 10:00:00 +0000https://www.chenshaowen.com/en/blog/custom-cluster-scheduler-to-optimize-tekton.html1. A Constrained Build Environment Cannot Meet Build Requirements Tekton is a CICD engine based on a Kubernetes cluster, and it is more cloud-native than Jenkins. In plain terms, that means it is easier to develop plugins for, easier to scale, easier to observe, and more fun. Because code can only be stored on the company intranet, the build cluster can only be deployed on the office intranet.微信公众号OptimizationTektonCICDDevOpsKubernetesOperationsLearningSchedulingBuildMulti-Clusterhttps://www.chenshaowen.com/en/blog/optimizing-the-slow-of-tekton-clone-task.htmlThu, 16 Feb 2023 00:00:00 +0000Thu, 16 Feb 2023 00:00:00 +0000https://www.chenshaowen.com/en/blog/optimizing-the-slow-of-tekton-clone-task.html1. Symptom - Tekton’s Clone Task Is Slow When running the clone task, Tekton is very time-consuming; with multiple repositories it generally takes around 2 minutes 30 seconds. As shown below: A pipeline that only clones takes 2 minutes 16 seconds to run, while the clone script itself actually runs for only 1-3 seconds.微信公众号TektonKubernetesCICDDevOpsOperationsPerformance Optimizationhttps://www.chenshaowen.com/en/blog/troubleshoot-slow-io-when-building-dockerfile.htmlThu, 02 Feb 2023 00:00:00 +0000Thu, 02 Feb 2023 00:00:00 +0000https://www.chenshaowen.com/en/blog/troubleshoot-slow-io-when-building-dockerfile.html1. The Problem Project details: File size 5.6 GB File count 529352 Dockerfile 1 2 3 FROM golang:1.13 COPY ./ /go/src/code The build command and its output are as follows: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 time DOCKER_BUILDKIT=1 docker build --no-cache -t test:v3 -f Dockerfile .微信公众号DockerImagesCICDBuildTroubleshootingOperationshttps://www.chenshaowen.com/en/blog/run-mv-is-slower-than-cp-in-dockerfile.htmlMon, 30 Jan 2023 00:00:00 +0000Mon, 30 Jan 2023 00:00:00 +0000https://www.chenshaowen.com/en/blog/run-mv-is-slower-than-cp-in-dockerfile.htmlUnlike on CentOS or Ubuntu, where we feel that mv is faster than cp, when you build an image with a Dockerfile, using Run cp is faster than Run mv. This post presents the test and verification data behind that. 1. Test Setup Machine environment Ubuntu 20.04.1 LTS 32C微信公众号DockerContainersCICDImagesOperationshttps://www.chenshaowen.com/en/blog/tekton-stress-test-and-optimize-k8s-cluster.htmlThu, 10 Nov 2022 00:00:00 +0000Thu, 10 Nov 2022 00:00:00 +0000https://www.chenshaowen.com/en/blog/tekton-stress-test-and-optimize-k8s-cluster.html1. Purpose of the Test Tune the parameters of the build cluster Probe the upper limit on the number of concurrent Tekton pipelines Give the optimal concurrency limit for a single cluster 2. Related Components and Machine Configuration Kubernetes version v1.21.4 Tekton version v0.24.1, kept consistent with production OpenEBS version localpv version 3.微信公众号TektonKubernetesStress TestingLarge ClustersOptimizationCICDOperationshttps://www.chenshaowen.com/en/blog/a-image-management-and-distribution-case-based-on-harbor-and-registry.htmlSat, 16 Jul 2022 00:00:00 +0000Sat, 16 Jul 2022 00:00:00 +0000https://www.chenshaowen.com/en/blog/a-image-management-and-distribution-case-based-on-harbor-and-registry.html1. The Challenge of a Cross-Region Harbor If all you need is to store image data in a simple way, Registry is an excellent choice as an image repository. Registry not only supports multiple storage backends, but can also be configured with HTTPS certificates and access credentials. Worth noting: Harbor微信公众号HarborRegistryImagesDistributionKubernetesOperationsLearningMulti-RegionAccelerationCICDhttps://www.chenshaowen.com/en/blog/how-to-insight-the-pipeline-of-tekton.htmlTue, 07 Jun 2022 00:00:00 +0000Tue, 07 Jun 2022 00:00:00 +0000https://www.chenshaowen.com/en/blog/how-to-insight-the-pipeline-of-tekton.html1. Scraping Tekton Metrics Add a ConfigMap configuration file 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 cat <<EOF | kubectl apply -f - apiVersion: v1 kind: ConfigMap metadata: name: config-observability namespace: tekton-pipelines labels: app.kubernetes.io/instance: default app.kubernetes.io/part-of: tekton-pipelines data: metrics.backend-destination: prometheus metrics.taskrun.level: "task" metrics.微信公众号KubernetesTektonMonitoringMetricsCICDOperationshttps://www.chenshaowen.com/en/blog/how-to-cache-third-party-package-whilie-building-in-stage.htmlTue, 22 Feb 2022 00:00:00 +0000Tue, 22 Feb 2022 00:00:00 +0000https://www.chenshaowen.com/en/blog/how-to-cache-third-party-package-whilie-building-in-stage.htmlIn a non-staged build scenario, when we use a container to build, we can mount the cache directory inside the container onto the build host, run the build task, and then copy the artifacts into the runtime image to produce the application image. But with a staged build, the build image and the runtime image live in the same Dockerfile, which makes it harder to optimize the caching of third-party dependencies.微信公众号CICDCIDevOpsCacheOptimizationhttps://www.chenshaowen.com/en/blog/develop-a-wechat-backend-using-tencent-serverless.htmlSat, 28 Aug 2021 00:00:00 +0000Sat, 28 Aug 2021 00:00:00 +0000https://www.chenshaowen.com/en/blog/develop-a-wechat-backend-using-tencent-serverless.html1. Where Serverless Fits If cloud computing is meant to make resources flow like tap water — turn it on and use it, turn it off and it stops — then Serverless is one of the future directions of cloud computing. Compared with runtimes such as IaaS and Kubernetes, Serverless微信公众号GitHubCICDServerlessWeChatR&DOperationshttps://www.chenshaowen.com/en/blog/a-cicd-system-based-on-tekton.htmlWed, 25 Aug 2021 00:00:00 +0000Wed, 25 Aug 2021 00:00:00 +0000https://www.chenshaowen.com/en/blog/a-cicd-system-based-on-tekton.html1. Background 1.1 Problems with the Current Use of Jenkins The orchestration engine is unstable Jenkins is an orchestration engine written in Java, and it stops the world (STW) during a full GC. Under large-scale builds, STW can prevent Jenkins from handling new requests. Heavy builds stall Jenkins stores data in disk files; every pipeline and every build occupies a file directory, producing a huge number of files.微信公众号CICDDevOpsTektonKubernetesOperationsPipelineLearninghttps://www.chenshaowen.com/en/blog/using-kubefed-to-distribute-tekton-resource-cross-cluster.htmlSat, 26 Jun 2021 00:00:00 +0000Sat, 26 Jun 2021 00:00:00 +0000https://www.chenshaowen.com/en/blog/using-kubefed-to-distribute-tekton-resource-cross-cluster.html1. The Advantages of Multi-Cluster Tekton Builds Thanks to Kubernetes, Tekton already has good elasticity and can support large-scale builds. At the same time, developing Tasks mainly uses Yaml and Shell, which widens the range of scenarios Tekton can adapt to. The figure above is a sketch of Tekton across multiple clusters.微信公众号TektonKubeFedKubernetesMulti-ClusterCICDOperationsLearningPipelineBuildDistributed Systemshttps://www.chenshaowen.com/en/blog/how-to-implement-approval-function-in-tekton.htmlThu, 24 Jun 2021 00:00:00 +0000Thu, 24 Jun 2021 00:00:00 +0000https://www.chenshaowen.com/en/blog/how-to-implement-approval-function-in-tekton.html1. Basic Features of a CICD Platform A common CICD engine is not suitable to hand directly to the business side. The main reasons are the high learning cost for users, the lack of necessary authentication, and the difficulty of maintenance and upgrades. We usually build on top of a process engine, adapting it to the business to improve usability and encapsulating it for specific scenarios to reduce complexity.微信公众号TektonKubernetesCICDDevOpsOperationshttps://www.chenshaowen.com/en/blog/how-to-customize-your-jenkins.htmlSat, 05 Jun 2021 00:00:00 +0000Sat, 05 Jun 2021 00:00:00 +0000https://www.chenshaowen.com/en/blog/how-to-customize-your-jenkins.html1. How to Deploy Jenkins To make the tests below easier, let me first introduce two ways to deploy Jenkins. Here I use the shaowenchen/jenkins:2.277.4 image. In a production environment, you should replace it with the official jenkins/jenkins image or your own custom image. 1.1 Running with docker-compose The docker-compose.yaml file微信公众号DevOpsCICDJenkinsDockerOperationshttps://www.chenshaowen.com/en/blog/a-universal-design-for-pipeline.htmlWed, 19 May 2021 00:00:00 +0000Wed, 19 May 2021 00:00:00 +0000https://www.chenshaowen.com/en/blog/a-universal-design-for-pipeline.html1. Decoupling the Engine to Unlock Pipeline Capabilities When designing a system, we often face a dilemma. Do we contain the complexity and offer a single, easy-to-use capability to the outside; or do we release the complexity and return flexibility to the user? This is a real test of product微信公众号Cloud NativeDevOpsCICDThoughtsR&Dhttps://www.chenshaowen.com/en/blog/how-to-add-physical-machines-to-tekton.htmlWed, 28 Apr 2021 00:00:00 +0000Wed, 28 Apr 2021 00:00:00 +0000https://www.chenshaowen.com/en/blog/how-to-add-physical-machines-to-tekton.html1. Why Physical Build Machines Are Needed In the article How to Add a Remote macOS Physical Machine for Jenkins Pipeline Builds, I described how to add a physical build machine in Jenkins. This was not a requirement I made up on a whim — at the time, real ToB commercial customers were actually asking for a solution.微信公众号TektonCloud NativeKubernetesCICDDevOpsOperationshttps://www.chenshaowen.com/en/blog/using-podman-to-build-images-under-kubernetes-and-jenkins.htmlThu, 25 Feb 2021 00:00:00 +0000Thu, 25 Feb 2021 00:00:00 +0000https://www.chenshaowen.com/en/blog/using-podman-to-build-images-under-kubernetes-and-jenkins.htmlIn principle, Jenkins can use Podman to build images in any Kubernetes cluster; this article uses Containerd as the example. 1. Dropping Docker Brings New Challenges to CICD In CICD scenarios, we often need to build and push images within a pipeline. In an earlier document, Creating Jenkins Slaves Dynamically on Kubernetes, I described how mounting the /var/run/docker.微信公众号JenkinsDevOpsCICDDockerKubernetesPodmanhttps://www.chenshaowen.com/en/blog/argocd-of-devops-tool-chain.htmlThu, 04 Feb 2021 00:00:00 +0000Thu, 04 Feb 2021 00:00:00 +0000https://www.chenshaowen.com/en/blog/argocd-of-devops-tool-chain.html1. What Problem Argo CD Solves 1.1 Starting from GitOps GitOps originated from a blog post published by Weaveworks in 2017: GitOps - Operations by Pull Request. In it, Alexis introduced a deployment approach that treats Git as the single source of truth. In GitOps practice, we need to define software infrastructure in a Git repository for management.微信公众号DevOpsArgoCDKubernetesCICDGitOpsOperationsLearningDeploymentBest Practiceshttps://www.chenshaowen.com/en/blog/10-tips-of-github-action.htmlSun, 31 Jan 2021 00:00:00 +0000Sun, 31 Jan 2021 00:00:00 +0000https://www.chenshaowen.com/en/blog/10-tips-of-github-action.htmlIf you already use GitHub Actions, this article will show you more interesting and useful ways to use it. By the time I finished writing, I had added several more workflows to my repositories. 1. Passing Parameters When a Workflow Runs When a workflow runs, you can allow parameters to be entered on the GitHub Actions page to control the execution logic.微信公众号GitHubGitHub ActionsCICDDevOpsR&Dhttps://www.chenshaowen.com/en/blog/a-debugger-for-actions.htmlMon, 18 Jan 2021 00:00:00 +0000Mon, 18 Jan 2021 00:00:00 +0000https://www.chenshaowen.com/en/blog/a-debugger-for-actions.html1. Debugging Until You Want to Quit GitHub Actions is the continuous integration service GitHub launched in October 2018. For open-source projects it offers unlimited free build time, and it supports Linux, MacOs, and Windows systems, which makes it very appealing. But a recent experience changed my view. I submitted a commit, improvement: build and ci, to a colleague’s repository, to improve its continuous build setup.微信公众号GitHubActionsDevOpsCICDdemohttps://www.chenshaowen.com/en/blog/the-concept-of-tekton-a-big-game.htmlSat, 12 Dec 2020 00:00:00 +0000Sat, 12 Dec 2020 00:00:00 +0000https://www.chenshaowen.com/en/blog/the-concept-of-tekton-a-big-game.html1. On the Aesthetics of CICD Tools In the document A Software Product Is the Output of a Team’s Capability, I mentioned that a software product is the delivery vehicle for a solution, and that its quality depends on the team’s understanding of the core problem. Only with a deep understanding of the domain can the delivered product have a chance of being good.微信公众号DevOpsTektonKubernetesCICDThoughtshttps://www.chenshaowen.com/en/blog/the-daemon-less-tools-of-kaniko.htmlFri, 11 Dec 2020 08:00:00 +0000Fri, 11 Dec 2020 08:00:00 +0000https://www.chenshaowen.com/en/blog/the-daemon-less-tools-of-kaniko.html1. daemon-less Image Build Tools 1.1 What Is a daemon-less Image Build Tool In a CICD pipeline, image builds come up often, and the conventional approach is to build with Docker in Docker or Docker out of Docker. For details, see: How to Use Docker in Docker In fact, to avoid monopolies and advance the industry, a unified OCI image format specification based on the Docker image format was defined long ago.微信公众号KanikoDevOpsImagesCICDKubernetesOperationshttps://www.chenshaowen.com/en/blog/how-to-add-self-hosted-runners-to-github-action.htmlFri, 04 Dec 2020 00:00:00 +0000Fri, 04 Dec 2020 00:00:00 +0000https://www.chenshaowen.com/en/blog/how-to-add-self-hosted-runners-to-github-action.html1. What GitHub Actions Is In an earlier post I put the CI capabilities GitLab provides into practice — click here to see. The advantages of using GitLab are that it can be deployed privately, it allows an unlimited number of private repositories, its CI configuration is simple, and it can connect to self-hosted Runners.微信公众号GitHubGitHub ActionsCICDDevOpsToolsR&DOperationshttps://www.chenshaowen.com/en/blog/capability-maturity-model-of-devops.htmlFri, 27 Nov 2020 00:00:00 +0000Fri, 27 Nov 2020 00:00:00 +0000https://www.chenshaowen.com/en/blog/capability-maturity-model-of-devops.htmlUnveiling the Capability Maturity Model of DevOps R&D Operations (DevOps) Solution Capability Rating Requirements - PDF Download Part 1: Overall Architecture - PDF Download Part 2: Agile Development Management Process - PDF Download Part 3: Continuous Delivery Process - PDF Download Part 4: Technical Operations - PDF Download Part 5: Application Design - PDF Download微信公众号DevOpsCICDStandardsModelshttps://www.chenshaowen.com/en/blog/how-to-run-the-pipeline-on-remote-osx.htmlSun, 22 Nov 2020 00:00:00 +0000Sun, 22 Nov 2020 00:00:00 +0000https://www.chenshaowen.com/en/blog/how-to-run-the-pipeline-on-remote-osx.htmlThis article also applies to connecting physical machines with ARM or MIPS architectures and FreeBSD or Windows systems. If Jenkins can reach the build machine, you can skip the Frp part. 1. The Problem In scenarios where Kubernetes is the infrastructure, when Jenkins runs a build pipeline it creates a separate Pod for each pipeline for building.微信公众号JenkinsmacOSCICDCIDevOpshttps://www.chenshaowen.com/en/blog/how-to-use-docker-in-docker.htmlSat, 21 Nov 2020 00:00:00 +0000Sat, 21 Nov 2020 00:00:00 +0000https://www.chenshaowen.com/en/blog/how-to-use-docker-in-docker.html1. Typical Use Cases In CI, there is usually a CI Engine responsible for parsing the pipeline and controlling the whole build process, while the actual build is delegated to an Agent. Jenkins and GitLab both work this way. As shown below, there are many kinds of Agents that connect to the CI Engine.微信公众号DockerKubernetesContainersCICICDOperationsLearningDinDBuildJenkinshttps://www.chenshaowen.com/en/blog/the-operation-guide-of-kubesphere-devops-3-0-pipeline.htmlWed, 14 Oct 2020 00:00:00 +0000Wed, 14 Oct 2020 00:00:00 +0000https://www.chenshaowen.com/en/blog/the-operation-guide-of-kubesphere-devops-3-0-pipeline.htmlThis post mainly covers how to operate and maintain a DevOps pipeline, and how to solve some common problems. The problems fall mainly into two categories: one is Kubernetes-related and fairly general in nature; the other is business-related and requires some understanding of the domain before you can solve problems微信公众号KubeSphereKubernetesDevOpsJenkinsOperationsCICDhttps://www.chenshaowen.com/en/blog/how-to-use-terraform-to-provide-iac-platform.htmlSat, 22 Aug 2020 00:00:00 +0000Sat, 22 Aug 2020 00:00:00 +0000https://www.chenshaowen.com/en/blog/how-to-use-terraform-to-provide-iac-platform.html1. Terraform Vs Kubernetes Infrastructure as Code (Iac) is built on immutable infrastructure: it uses orchestration tools to turn infrastructure into text, so you can manage infrastructure the way you manage code. In 2018 I was working on SaaS development, deploying on the Kubernetes platform, and that was the year Terraform was very hot.微信公众号TerraformIacKubernetesDevOpsCICDR&Dhttps://www.chenshaowen.com/en/blog/jenkins-x-is-not-jenkins-but-stack.htmlSun, 16 Aug 2020 00:00:00 +0000Sun, 16 Aug 2020 00:00:00 +0000https://www.chenshaowen.com/en/blog/jenkins-x-is-not-jenkins-but-stack.html1. Introduction to Jenkins X Relying on its massive plugin ecosystem, Jenkins currently holds most of the enterprise CICD engine market share. But in the cloud native era, Jenkins has also exposed many problems: a single point of service, disk storage, memory usage, and so on. Around Kubernetes, Jenkins X offers a DevOps approach better suited to the cloud native era.微信公众号Tech StackJenkinsJenkins-XKubernetesDevOpsCICDOperationshttps://www.chenshaowen.com/en/blog/the-stress-test-about-kubernetes-dynamically-creates-jenkins-agent.htmlSun, 02 Aug 2020 00:00:00 +0000Sun, 02 Aug 2020 00:00:00 +0000https://www.chenshaowen.com/en/blog/the-stress-test-about-kubernetes-dynamically-creates-jenkins-agent.htmlIn the previous document, we took advantage of the elasticity Kubernetes provides to dynamically create Jenkins Slaves on Kubernetes. This document is mainly a stress test of Jenkins under large-scale builds. 1. Cluster Configuration 1.1 Kubernetes Version The version used here is v1.16.7 1 2 3 4 kubectl version Client Version: version.微信公众号JenkinsKubernetesDevOpsCICDTestingOperationshttps://www.chenshaowen.com/en/blog/book/the-essentials-of-devops.htmlTue, 23 Jun 2020 00:00:00 +0000Tue, 23 Jun 2020 00:00:00 +0000https://www.chenshaowen.com/en/blog/book/the-essentials-of-devops.htmlAuthor: (Russia) Oleg Skrynnik Publisher: Tsinghua University Press Publication year: 2020-05-01 ISBN: 9787302547143 Notes: As the title suggests, the book focuses on DevOps from a business perspective. From its origins to its foundations, from principles to practice, you can take something away at every level. DevOps rests on lean production (the Toyota production line) and agile development, with the value chain at its core.微信公众号BooksDevOpsCICDR&Dhttps://www.chenshaowen.com/en/blog/how-to-develop-the-plugin-of-jenkins.htmlSat, 20 Jun 2020 00:00:00 +0000Sat, 20 Jun 2020 00:00:00 +0000https://www.chenshaowen.com/en/blog/how-to-develop-the-plugin-of-jenkins.htmlUsing Jenkins always means dealing with all kinds of plugins. To practice DevOps better, we should also be capable of developing plugins ourselves, so the whole process can converge inside Jenkins. 1. Jenkins Plugins 1.1 The Plugin Ecosystem Jenkins’ predecessor Hudson started in 2004, and 16 years on it is still a mainstream CI/CD engine.微信公众号JenkinsDevOpsCICDR&DPluginJavaLearninghttps://www.chenshaowen.com/en/blog/prow-of-devops-tool-chain.htmlFri, 17 Apr 2020 13:17:44 +0000Fri, 17 Apr 2020 13:17:44 +0000https://www.chenshaowen.com/en/blog/prow-of-devops-tool-chain.html1. About Prow In the GitHub repositories of well-known projects such as Kubernetes and Istio, we often see an xxx-bot user adding labels to issues and merging PRs. That bot account is driven by Prow. Prow is a project of the Kubernetes Testing Special Interest Group and is currently part of kubernetes/test-infra.微信公众号DevOpsProwKubernetesCICDhttps://www.chenshaowen.com/en/blog/build-an-automated-test-system-using-kubernetes-and-jenkins.htmlThu, 12 Dec 2019 00:00:00 +0000Thu, 12 Dec 2019 00:00:00 +0000https://www.chenshaowen.com/en/blog/build-an-automated-test-system-using-kubernetes-and-jenkins.html1. Test Layering The purpose of testing is to verify expected behavior and uncover latent defects. Testing strengthens confidence that a qualified product can be delivered, and it is what makes agile iteration possible. You could say that testing determines the development pace of a product. The network model has the seven-layer OSI and the four-layer TCP, while development patterns include MTV, MVC, MVP, MVVM, and so on.微信公众号TestingAutomationKubernetesJenkinsDevOpsCICDOperationshttps://www.chenshaowen.com/en/blog/creating-jenkins-slave-dynamically-on-kubernetes.htmlFri, 06 Dec 2019 00:00:00 +0000Fri, 06 Dec 2019 00:00:00 +0000https://www.chenshaowen.com/en/blog/creating-jenkins-slave-dynamically-on-kubernetes.html1. Jenkins Working Modes Jenkins has a single-Master, multi-Slave architecture. The Master assigns tasks and manages services. Slaves execute the actual tasks. Even when multiple Masters are deployed, they remain independent of one another and cannot coordinate scheduling. A high-availability Jenkins solution requires an external task distribution framework, such as微信公众号JenkinsKubernetesCICDDevOpsOperationshttps://www.chenshaowen.com/en/blog/sonarqube-of-code-quality-analysis-tool.htmlWed, 21 Aug 2019 00:00:00 +0000Wed, 21 Aug 2019 00:00:00 +0000https://www.chenshaowen.com/en/blog/sonarqube-of-code-quality-analysis-tool.html1. Who Needs SonarQube Code review is an indispensable part of high-quality software development. It helps developers spot bugs in the code in time and improves code quality and maintainability. The dimensions of code review include language conventions, code style, and soundness of design. Checking these things by hand consumes微信公众号SonarQubeDevOpsCICDR&DCode Qualityhttps://www.chenshaowen.com/en/blog/how-to-use-jenkins-docker-gitlab-to-build-django-automated-deployment-process.htmlSun, 25 Nov 2018 00:00:00 +0000Sun, 25 Nov 2018 00:00:00 +0000https://www.chenshaowen.com/en/blog/how-to-use-jenkins-docker-gitlab-to-build-django-automated-deployment-process.htmlOne of the illusions that programmers at big companies easily fall into is mistaking platform capabilities for their own abilities. In a large team, we should not focus only on our own little patch of ground; we need to understand every part of the platform. On one hand, this helps us make better use of the platform’s features; on the other, it helps our own technical growth.微信公众号DjangoGitLabJenkinsDockerDevOpsCICDR&DOperationshttps://www.chenshaowen.com/en/blog/how-to-package-a-docker-image-of-robot-framework.htmlTue, 18 Sep 2018 00:00:00 +0000Tue, 18 Sep 2018 00:00:00 +0000https://www.chenshaowen.com/en/blog/how-to-package-a-docker-image-of-robot-framework.htmlTo make it easier for CI to integrate UI automation tests, the Robot Framework runtime needs to be packaged as a Docker image. This post covers the configuration and scripts involved in the packaging process. 1. Packaging Directory Structure 1 2 3 4 5 6 tree . ├──微信公众号DockerRobotFrameworkPythonTestingCICDOperationshttps://www.chenshaowen.com/en/blog/basic-of-robot-framework.htmlTue, 24 Jul 2018 00:00:00 +0000Tue, 24 Jul 2018 00:00:00 +0000https://www.chenshaowen.com/en/blog/basic-of-robot-framework.htmlThis post mainly introduces some basic concepts of Robot Framework and how to write a test case. 1. Basic Elements 1.1 Keywords Robot Framework keywords are similar to functions. They are divided into system keywords and user-defined keywords. System keywords, imported by loading a Library User keywords, imported by loading微信公众号PythonTestingRobotFrameworkDevOpsLearningAutomated TestingR&DCICDhttps://www.chenshaowen.com/en/blog/front-webpack-practice-of-gitlab-ci.htmlWed, 08 Nov 2017 09:15:56 +0000Wed, 08 Nov 2017 09:15:56 +0000https://www.chenshaowen.com/en/blog/front-webpack-practice-of-gitlab-ci.htmlStarting with GitLab 8.0, GitLab began integrating CI (Continuous Integration) functionality. You only need to add a Runner on the server and a .gitlab-ci.yml file in the project, and you can run CI. In GitLab Setup and Configuration I recorded the whole process of setting up a GitLab service from微信公众号FrontendContinuous IntegrationToolsDevOpsGitLabCICDhttps://www.chenshaowen.com/en/blog/jenkins-pipeline-usging-and-debug.htmlMon, 14 Aug 2017 13:13:55 +0000Mon, 14 Aug 2017 13:13:55 +0000https://www.chenshaowen.com/en/blog/jenkins-pipeline-usging-and-debug.html1. Basic Concepts master master is where Jenkins is installed and runs; it is responsible for parsing job scripts, handling tasks, and scheduling compute resources. agent agent is responsible for handling tasks dispatched from master; the actual operations are executed through an executor. executor An executor is the compute resource微信公众号JenkinsPipelineContinuous IntegrationDevOpsCICDDebuggingR&D