1. Calico
1.1 BIRD is not ready
1
| kubectl -n kube-system get pod
|
calico-node-xxx stays at 0/1 and never comes up, reporting calico/node is not ready: BIRD is not ready: BGP not established with
Solution:
Calico uses first-found by default, meaning it takes the NodeIP from the first NIC it finds. Although NICs such as lo and docker0 are excluded, detection can still fail with some probability. You need to change it manually and specify the NIC.
- View the NICs on the host
1
2
3
4
5
6
7
8
9
10
11
12
13
14
| ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:50:56:92:3a:20 brd ff:ff:ff:ff:ff:ff
inet 10.13.5.65/23 brd 10.13.5.255 scope global ens160
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:fe92:3a20/64 scope link
valid_lft forever preferred_lft forever
|
- Edit the Calico deployment
1
| kubectl -n kube-system edit ds calico-node
|
Just set the interface in IP_AUTODETECTION_METHOD to the NIC name; wildcards are supported.
1
2
3
4
5
6
| spec:
containers:
- env:
- name: IP_AUTODETECTION_METHOD
value: interface=ens160
image: docker.io/calico/node:v3.18.2
|
2. Metric Server
2.1 Cannot access Metric Server
The Metric Server service is unreachable.
Solution:
1
| kubectl -n kube-system edit deploy metrics-server
|
Modify the startup arguments:
1
2
| - --kubelet-insecure-tls
- --kubelet-preferred-address-types=InternalIP
|
Skip certificate verification and communicate using the Node’s IP.
3. NFS Storage
3.1 selfLink was empty
In the NFS Pod you will see error logs like the following
1
2
| I0916 06:12:44.587396 1 leaderelection.go:185] attempting to acquire leader lease default/cluster.local-nfs-client-nfs-client-provisioner...
E0916 06:12:44.597222 1 event.go:259] Could not construct reference to: '&v1.Endpoints{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster.local-nfs-client-nfs-client-provisioner", GenerateName:"", Namespace:"default", SelfLink:"", UID:"bd270086-5338-464b-b50d-b3ec110fc6d1", ResourceVersion:"4413847", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63767369564, loc:(*time.Location)(0x1956800)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string{"control-plane.alpha.kubernetes.io/leader":"{\"holderIdentity\":\"nfs-client-nfs-client-provisioner-5d4fd84f8b-pnwkv_1b60c33d-16b5-11ec-9098-dae3d1cb24d6\",\"leaseDurationSeconds\":15,\"acquireTime\":\"2021-09-16T06:12:44Z\",\"renewTime\":\"2021-09-16T06:12:44Z\",\"leaderTransitions\":0}"}, OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Subsets:[]v1.EndpointSubset(nil)}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'LeaderElection' 'nfs-client-nfs-client-provisioner-5d4fd84f8b-pnwkv_1b60c33d-16b5-11ec-9098-dae3d1cb24d6 became leader'
|
Solution:
Starting with Kubernetes 1.20, the metadata.selfLink field was removed by default. But nfs-client-provisioner still uses that field. Therefore it must be enabled in kube-apiserver.
Edit /etc/kubernetes/manifests/kube-apiserver.yaml and add the line - --feature-gates=RemoveSelfLink=false to the startup arguments.
