Hi all,
I am running a Kubernetes cluster (Calico + IPVS kube-proxy). Pods can ping CoreDNS pods directly, but DNS queries to CoreDNS (both Pod IPs and ClusterIP 10.233.0.3) always time out.
- CoreDNS pods are running
- ping to CoreDNS Pod IPs works, but dig/nslookup to UDP/TCP 53 on Pod IPs and ClusterIP fail.
- iptables/ipvsadm shows services for 10.233.0.3:53 with the correct endpoints.
Has anyone seen this issue where CoreDNS is reachable via ICMP but not over DNS (UDP/TCP 53)? Any tips would be appreciated.
I don’t think that pods respond to ICMP in general. Try checking the pods using kubectl; what do you see when you do k get -n kube-system get pods? Are the pods up and in a good state? Also, how are you checking DNS? You’ll need to do something like
k run busy --image busybox -- sleep 60; k exec busy -- nslookup checkme.com
Hello,
here is the result from nslookup
kubectl exec busy -- nslookup checkme.com ─╯
Server: 169.254.25.10
Address: 169.254.25.10:53
Non-authoritative answer:
Name: checkme.com
Address: 62.219.91.45
Non-authoritative answer:
kubectl get pods -n kube-system ─╯
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-dbf6dc49-zlm94 1/1 Running 0 9d
calico-node-5gz9q 1/1 Running 0 3d15h
calico-node-6whfc 1/1 Running 0 3d15h
calico-node-gjnpt 1/1 Running 0 3d15h
calico-node-h55vh 1/1 Running 0 3d15h
calico-node-hhx8c 1/1 Running 0 3d15h
calico-node-m9fxz 1/1 Running 0 3d15h
coredns-786c68fb56-49fqq 1/1 Running 0 2d19h
coredns-786c68fb56-r4qhm 1/1 Running 0 2d19h
dns-autoscaler-56cb45595c-5wj6d 1/1 Running 0 9d
kube-apiserver-control-plane-1 1/1 Running 3 9d
kube-apiserver-control-plane-2 1/1 Running 3 9d
kube-apiserver-control-plane-3 1/1 Running 3 9d
kube-controller-manager-control-plane-1 1/1 Running 5 9d
kube-controller-manager-control-plane-2 1/1 Running 5 9d
kube-controller-manager-control-plane-3 1/1 Running 5 9d
kube-proxy-99zdv 1/1 Running 0 9d
kube-proxy-9vw7g 1/1 Running 0 9d
kube-proxy-hhjxk 1/1 Running 0 9d
kube-proxy-mvbfh 1/1 Running 0 3d23h
kube-proxy-rm8cc 1/1 Running 0 3d23h
kube-proxy-tmjc8 1/1 Running 0 9d
kube-scheduler-control-plane-1 1/1 Running 3 9d
kube-scheduler-control-plane-2 1/1 Running 3 9d
kube-scheduler-control-plane-3 1/1 Running 3 9d
kubernetes-dashboard-599bcc4fc7-clvt5 1/1 Running 0 7d19h
kubernetes-metrics-scraper-6bb6474655-hj2vj 1/1 Running 0 7d19h
metrics-server-66fd4b45-2ssws 1/1 Running 0 8d
metrics-server-66fd4b45-6j6k4 1/1 Running 0 8d
metrics-server-66fd4b45-ff24d 1/1 Running 0 8d
nginx-proxy-worker-node-1 1/1 Running 1 9d
nginx-proxy-worker-node-2 1/1 Running 1 9d
nginx-proxy-worker-node-3 1/1 Running 1 9d
nodelocaldns-g5cz6 1/1 Running 1 (7d1h ago) 7d1h
nodelocaldns-k5q7z 1/1 Running 0 7d1h
nodelocaldns-lnngv 1/1 Running 0 7d1h
nodelocaldns-ndtn5 1/1 Running 0 7d1h
nodelocaldns-nrb8x 1/1 Running 0 7d1h
nodelocaldns-wzh98 1/1 Running 0 7d1h