Kubelet service is in failed state while setting up the k8 cluster version is latest version of Kubernetes 1.23

Kubernetes-Slack Discussions
#1

When I tried to install the kubelet service while create K8 cluster on Amazon linux 2 machine it is showing in failed sate and getting the given error when tried to debug the service:

Feb 10 04:53:32 ip-172-31-90-161.ec2.internal systemd[1]: kubelet.service holdoff time over, scheduling restart.
Feb 10 04:53:32 ip-172-31-90-161.ec2.internal systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
Feb 10 04:53:32 ip-172-31-90-161.ec2.internal systemd[1]: Started kubelet: The Kubernetes Node Agent.
Feb 10 04:53:32 ip-172-31-90-161.ec2.internal kubelet[3720]: E0210 04:53:32.344649 3720 server.go:205] “Failed to load kubelet config file” err="failed to load Kubelet config file /var/lib/kubelet/config.yaml, error failed to read kubelet config file
Feb 10 04:53:32 ip-172-31-90-161.ec2.internal systemd[1]: kubelet.service: main process exited, code=exited, status=1/FAILURE
Feb 10 04:53:32 ip-172-31-90-161.ec2.internal systemd[1]: Unit kubelet.service entered failed state.
Feb 10 04:53:32 ip-172-31-90-161.ec2.internal systemd[1]: kubelet.service failed.
Feb 10 04:53:42 ip-172-31-90-161.ec2.internal systemd[1]: kubelet.service holdoff time over, scheduling restart.
Feb 10 04:53:42 ip-172-31-90-161.ec2.internal systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
Feb 10 04:53:42 ip-172-31-90-161.ec2.internal systemd[1]: Started kubelet: The Kubernetes Node Agent.
Feb 10 04:53:42 ip-172-31-90-161.ec2.internal kubelet[3727]: E0210 04:53:42.594855 3727 server.go:205] “Failed to load kubelet config file” err="failed to load Kubelet config file /var/lib/kubelet/config.yaml, error failed to read kubelet config file
Feb 10 04:53:42 ip-172-31-90-161.ec2.internal systemd[1]: kubelet.service: main process exited, code=exited, status=1/FAILURE
Feb 10 04:53:42 ip-172-31-90-161.ec2.internal systemd[1]: Unit kubelet.service entered failed state.
Feb 10 04:53:42 ip-172-31-90-161.ec2.internal systemd[1]: kubelet.service failed.
Feb 10 04:53:52 ip-172-31-90-161.ec2.internal systemd[1]: kubelet.service holdoff time over, scheduling restart.
Feb 10 04:53:52 ip-172-31-90-161.ec2.internal systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
Feb 10 04:53:52 ip-172-31-90-161.ec2.internal systemd[1]: Started kubelet: The Kubernetes Node Agent.
Feb 10 04:53:52 ip-172-31-90-161.ec2.internal kubelet[3735]: E0210 04:53:52.841156 3735 server.go:205] “Failed to load kubelet config file” err="failed to load Kubelet config file /var/lib/kubelet/config.yaml, error failed to read kubelet config file
Feb 10 04:53:52 ip-172-31-90-161.ec2.internal systemd[1]: kubelet.service: main process exited, code=exited, status=1/FAILURE
Feb 10 04:53:52 ip-172-31-90-161.ec2.internal systemd[1]: Unit kubelet.service entered failed state.
Feb 10 04:53:52 ip-172-31-90-161.ec2.internal systemd[1]: kubelet.service failed.
Feb 10 04:54:03 ip-172-31-90-161.ec2.internal systemd[1]: kubelet.service holdoff time over, scheduling restart.

image
image1480×934 161 KB

When I check the /var/lib/kubelet it don’t have the kubelete directory and the config.yaml file I used the kubernetes.io tech note to install the k8 cluster .

#2

Hello Bansod.ashish

Did you run the kubeadm init on the master and join command on the worker node? and what is the output of kubectl get nodes and kubectl get pods -n kube-system ?

Thanks,
KodeKloud Support

#3

In my case the problem was related to cgroup.
I run cat /var/lib/syslog , there was a log saying Failed to run kubelet" err="failed to run Kubelet: misconfiguration: kubelet cgroup driver: \"systemd\" is different from docker cgroup driver: \"cgroupfs\""
Solution here help to solve issue.

#4

You can find similar solution from the official docs also: -