Kubernet deployment and connection refused on the master after 3-4 min from initializing the control-plane node

Kubernetes
#1

Hi

I am getting into very strange issue as I m deploying K8s 1.31 on Ubuntu 22.04

Once kubeadm init is completed on the master, I follow the instructions to copy the admin.conf to my home folder, etc… and before deploying the network add-on like Weave net

In the first 3-4 mins, I can communicate with the API Server and do things like kubectl get nodes

After that, I start getting :
The connection to the server 192.168.110.100:6443 was refused - did you specify the right host or port?

Has anyone seen this before? Thank you

#2

Sounds like you have not correctly configured containerd.

See step 5 on this document.

It needs to be done on all nodes

#3

Thank you very much Alistair; interesting those steps are not in the official documentation here

#4

It is there, this is it, but that’s the do it by hand version, so if you mess up that edit then it will break.

The way I have documented it gets sed to do the edit automatically which reduces the chance of user error.

Assuming you did the edit manually

  1. Delete the config.toml file you created
  2. sudo systemctl restart containerd - so it “forgets” the previous setting
  3. Redo the step as per the first link I sent.
#5

Okay, I started over and decided to ditch the official documentation. I used the link you sent me to set the nodes and initialize the master. I followed all the steps, and it just pukes.

In the video, the instructor follows the documentation at the time of the recording (
Demo - Kubeadm - Part 2 - Configure Cluster with kubeadm within the Kubernetes for the Absolute Beginners - Hands-on Tutorial class) , but the current documentation is different now as a few commands are missing compared to the recording, like the one I mentioned with the overlay and br_netfilter kernel modules.

[kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s
[kubelet-check] The kubelet is not healthy after 4m0.000619017s

Unfortunately, an error has occurred:
        The HTTP call equal to 'curl -sSL http://127.0.0.1:10248/healthz' returned error: Get "http://127.0.0.1:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused


This error is likely caused by:
        - The kubelet is not running
        - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
        - 'systemctl status kubelet'
        - 'journalctl -xeu kubelet'

Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
        - 'crictl --runtime-endpoint unix:///var/run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
        Once you have found the failing container, you can inspect its logs with:
        - 'crictl --runtime-endpoint unix:///var/run/containerd/containerd.sock logs CONTAINERID'
error execution phase wait-control-plane: could not initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
#6

I think I’m in the clear now.

So, I followed the recording but did steps 2, 3, and 5 only from your GitHub document within the Node Setup section instead of what was mentioned in the recording and the official documentation.

Thank you for your help and pointing out the root of the issue

#7

The videos for cluster setup are out of date and will be re-done when the CKA course is revised for the new exam material. The most up to date install instructions are the github pages.

#8

That is good to know. Thank you again!