Hello everyone. I'm trying to deploy a k8s cluster using `kubeadm` with `contain . . .

Kubernetes-Slack Discussions
#1

Timekiller:
Hello everyone. I’m trying to deploy a k8s cluster using kubeadm with containerd as the container runtime on an AWS EC2 instance with Amazon Linux 2 and coming across the following problem:

I’ve been following along through this demo lesson https://kodekloud.com/topic/demo-deployment-with-kubeadm/ with the documentation here: https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/

Everything has been going well so far but I seem to be stuck on my CoreDNS pods being in ContainerCreating state:

NAME                                                   READY   STATUS              RESTARTS       AGE
coredns-6d4b75cb6d-27hwl                               0/1     ContainerCreating   0              19m
coredns-6d4b75cb6d-rdst6                               0/1     ContainerCreating   0              19m

Looking at the event logs it seems like this issue is failed to find network info for sandbox which looks similar to what @Abdullah Naeem is experiencing.

Events:
  Type     Reason                  Age                 From               Message
  ----     ------                  ----                ----               -------
  Normal   Scheduled               21m                 default-scheduler  Successfully assigned kube-system/coredns-6d4b75cb6d-27hwl to ip-172-31-28-82.ec2.internal
  Warning  FailedCreatePodSandBox  20m                 kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "ccd66db5747fbb1b0d170c979f4dece6b18a024dbe0b464c5a7c684aeb4bf01c": failed to find network info for sandbox "ccd66db5747fbb1b0d170c979f4dece6b18a024dbe0b464c5a7c684aeb4bf01c"
  Warning  FailedCreatePodSandBox  57s (x81 over 18m)  kubelet            (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "e14e36d7fbd228f38c12415872d16b800a30458c089c89ff77f40066aa6e94ef": failed to find network info for sandbox "e14e36d7fbd228f38c12415872d16b800a30458c089c89ff77f40066aa6e94ef"

Any clues? I can provide an exact log of every single step/command taken to reach this point, as I’m provisioning the VM with IaC tools - rebuilding the VM always results in this same result, so I’m pretty sure I missed something somewhere.

#2

Timekiller:
Also willing to hop on a call with someone and display the whole setup and the issue in great detail :slightly_smiling_face:

#3

Matthew Robinson:
What network CNI layer are you using?

#4

Timekiller:
Oh hey Matthew! Fancy meeting you here :slightly_smiling_face:

The same one being used in the demo: https://www.weave.works/docs/net/latest/kubernetes/kube-addon/

#5

Timekiller:
Does weave net expect the docker container engine? I just noticed this in the weave logs:

INFO: 2022/06/01 05:22:22.457418 Command line options: map[conn-limit:200 datapath:datapath db-prefix:/weavedb/weave-net docker-api: expect-npc:true http-addr:127.0.0.1:6784 ipalloc-init:consensus=0 ipalloc-range:10.32.0.0/12 metrics-addr:0.0.0.0:6782 name:2e:59:61:6c:f0:bd nickname:ip-172-31-28-82.ec2.internal no-dns:true no-masq-local:true port:6783]

I’m using containerd

#6

Matthew Robinson:
I just had a bit of a Google and it looks like a few people are having issues with containerd and WeaveNet

#7

Matthew Robinson:
https://serverfault.com/questions/1100808/pod-coredns-stuck-in-containercreating-state-with-weave-on-k8s

#8

Matthew Robinson:
https://github.com/containerd/containerd/issues/6921

#9

Timekiller:
Yeah, I’ve seen that first one.

#10

Timekiller:
Any alternative CNI you’d recommend? There are a bunch

#11

Timekiller:
> https://github.com/containerd/containerd/issues/6921
I think you found it!

#12

Timekiller:
I’m using the same version of containerd as well 1.6.4

#13

Matthew Robinson:
When I started playing I used flannel as it is probably the simplest of the CNIs

#14

Timekiller:
That’s an excellent suggestion. Thank you so much! I’ll do that next :slightly_smiling_face:

#15

Matthew Robinson:
If I remember correctly flannel can’t do some things, network policies rings a bell

#16

Timekiller:
That’s good to keep in mind! I can always switch later if flannel works

#17

Matthew Robinson:
I’m mostly using EKS at the moment with the AWS VPC CNI which I wouldn’t recommend for experimenting

#18

Timekiller:
No worries, I’ll stay away from EKS! Still planning to move to a more managed solution later on after I’m done playing with kubeadm setup

#19

Timekiller:
That issue is now solved! Thanks again Matthew! :slightly_smiling_face:

image.png
image.png1173×477 147 KB

#20

unnivkn:
Hi @Timekiller the root cause here is while you install kubeadm you might have passed --pod-network-cidr= <some IP Rang>. So this Ip range should match with the default IP range of the CNI you are trying to install. For eg: weavenetwork use 10.x.x.x Ip range & Flannel uses 172.x.x.x IP range. So if you use Flannel Ip range(eg: 172.1.0.22) along with kubadm command & try to install weave network CNI you may get the error, which you already encountered above. fyr:

pod-network-cidr-1.PNG
pod-network-cidr-1.PNG932×134 19.1 KB

pod-network-cidr-2.PNG
pod-network-cidr-2.PNG961×524 58.1 KB

pod-network-cidr-3.PNG
pod-network-cidr-3.PNG743×595 15.4 KB