Troubleshooting
Kubernetes Cluster Troubleshooting
· ☕ 6 min read
FailedCreatePodSandBox Error Error response from daemon: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:402: getting the final child's pid from pipe caused: EOF: unknown Resolution Clear the cache 1 echo 3 > /proc/sys/vm/drop_caches Cause Too much memory fragmentation calico-node Keeps Restarting increase max user Error runtime: failed to

How to Repair a Kubernetes Cluster After Changing IPs
· ☕ 3 min read
A record of repairing a cluster failure caused by an IP change. There are two clusters: one single-node (allinone) cluster, and one four-node cluster (3 masters, 1 node). 1. Update the Etcd Certificates [On every Etcd node] Back up the Etcd certificates 1 cp -R /etc/ssl/etcd/ssl /etc/ssl/etcd/ssl-bak View the domains

Traffic Restriction Using Linux TC
· ☕ 5 min read
1. Traffic Control Principles Under Linux By queuing packets, we can control how data packets are sent. This kind of control is called data shaping (shape the data), and it includes the following operations on data: Adding latency Dropping packets Reordering Duplicating, corrupting Rate control Under the qdisc-class-filter structure, controlling traffic requires three steps:

Adding SSH Key Login and Troubleshooting
· ☕ 6 min read
1. Steps to Add a Key 1.1 Generate an SSH Key on the Client There are two key formats: Old format, the private key begins with -----BEGIN RSA PRIVATE KEY----- 1 ssh-keygen -m PEM -t rsa -b 4096 -C "mail@chenshaowen.com" New format, the private key begins with -----BEGIN OPENSSH PRIVATE KEY----- 1 ssh-keygen -t rsa -b 4096 -C "mail@chenshaowen.

Common Commands for Cleaning Up Kubernetes Cluster Resources
· ☕ 4 min read
Clusters that run for a long time often face various kinds of resource exhaustion. On top of that, when disk space runs low, Kubelet will proactively clean up images on its own, adding another source of uncertainty. This article provides some command snippets for cleanup work. 1. Cleaning Up Basic Kubernetes Objects Clean up Pods in Evicted state 1 sudo kubectl get pods --all-namespaces -o wide | grep Evicted | awk '{print $1,$2}' | sudo xargs -L1 kubectl delete pod -n Clean up Pods in Error state 1 sudo kubectl get pods --all-namespaces -o wide | grep Error | awk '{print $1,$2}' | sudo xargs -L1 kubectl delete pod -n Clean up Pods in Completed state 1 sudo kubectl get pods --all-namespaces -o wide | grep Completed | awk '{print $1,$2}' | sudo xargs -L1 kubectl delete pod -n Clean up unused PVs 1 sudo kubectl describe -A pvc | grep -E "^Name:.