Etc/containerd/config.toml does not work correctly

Kubernetes
#1

Hey everyone,

I hope you’re all doing well.

I’ve been working on setting up a multi-node cluster using the Kubeadm tool. Everything seemed to be going smoothly until I ran into an issue with the kubectl command not connecting to the kubeapi server. Upon further investigation using the crictl tool, I noticed that the pods were constantly in an exited state and kept rebooting.

In my search for a solution, I discovered that the configuration provided by the Kubernetes documentation and the CKA lecture video was not working as expected. Specifically, the following configuration snippet:

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
  ...
  [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
    SystemdCgroup = true

Instead, I stumbled upon a version 2 configuration that resolved my issue:

version = 2
[plugins]
  [plugins."io.containerd.grpc.v1.cri"]
   [plugins."io.containerd.grpc.v1.cri".containerd]
      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes]
        [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
          runtime_type = "io.containerd.runc.v2"
          [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
            SystemdCgroup = true

I implemented this configuration and it resolved my issue. However, I must admit that I don’t fully understand why it worked or where else this configuration might be documented, as I only found it in GitHub issues.

If anyone can shed some light on this, I’d greatly appreciate it. Thanks!

#2

If the V2 config works for you, it works. So good that.
If you want to try a more up-to-date configuration, take a look at the tutorial from the CKA repo. The Node Setup page has a procedure for setting up config.toml that is a bit easier to follow and get right than the procedure from the course video.

#3

Hi,

Thank you for the guide. I tried to follow the same steps but the nodes are not coming up.

NAME STATUS ROLES AGE VERSION
masternode NotReady control-plane 16m v1.25.16
workernode1 NotReady 6m19s v1.25.16
workernode2 Ready 5m24s v1.25.16
vganji@masternode:/etc/cni/net.d$

I see the below logs

Feb 17 08:51:31 masternode kubelet[14560]: E0217 08:51:31.402728 14560 kubelet.go:2373] “Container runtime network not ready” networkReady=“NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized”
Feb 17 08:51:36 masternode kubelet[14560]: E0217 08:51:36.404680 14560 kubelet.go:2373] “Container runtime network not ready” networkReady=“NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized”
Feb 17 08:51:41 masternode kubelet[14560]: E0217 08:51:41.406075 14560 kubelet.go:2373] “Container runtime network not ready” networkReady=“NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized”

ls /etc/cni/net.d/
10-weave.conflist

Please help on how to fix this

Thanks,
Vamsi

#4

Restarting containerd, solved the issue

Thank you