Hi, I've question: In CKA course's ETCD lab (<https://kodekloud.com/topic/soluti . . .

Kubernetes-Slack Discussions
#1

Taimoor Mirza:
Hi, I’ve question: In CKA course’s ETCD lab (https://kodekloud.com/topic/solution-backup-and-restore/), there’s task related with restoring ETCD. Whenever I restored the data store and modified the volume path in ETCD’s manifest, it crashed the kube-apiserver. Kubectl always throws error related with port 6443, connection refused. In the solution video, everything goes smoothly. Any idea why?

FYI, I applied the following steps in the lab:

  1. restore data store
etcdctl restore snapshot.db --data-dir=/var/lib/etcd-from-backup
  1. modify path of etcd-volume in ETCD manifest (/etc/kubernetes/manifest/etcd.yaml)
 volumes:
    - hostPath:
      path: /var/lib/etcd-from-backup
      type: DirectoryOrCreate
    name: etcd-data
#2

Sara:
Hi, as far as i remember i edited this path at three places. it seems you have only changed hostpath.

#3

Taimoor Mirza:
In the solution video as well in other tutorials (https://github.com/bmuschko/cka-crash-course/blob/master/exercises/04-etcd-backup-restore/solution/solution.md#:~:text=To%20restore%20etcd%20from%20the,%2Flib%2Ffrom-backup%20.), the instructor/author changes only the value of volume’s path.

In my case, not only the kube-apiserver refuses to connect but the ETCD Pod doesn’t restart either.

#4

Ajay:
path change in volumes: (hostPath) is enough because in volumeMounts we use the volume name

#5

Taimoor Mirza:
then why doesn’t ETCD pod restart? docker ps | grep etcd shows nothing and kube-apiserver throws connection refusal error.

#6

unnivkn:
Hi @Taimoor Mirza please stop & start etcd by relocating it to another directory and bring it back.

#7

unnivkn:
eg:
mv /etc/kubernetes/manifest/etcd.yaml /etc/kubernetes/etcd.yaml
k -n kube-system get po | grep -i etcd
mv /etc/kubernetes/etcd.yaml /etc/kubernetes/manifest/etcd.yaml
k -n kube-system get po | grep -i etcd

#8

Shanmuga Raj:
You need to follow these steps, so you can restore the DB to the original folder /var/lib/etcd:

  1. mkdir backups - in your home path
  2. mv /etc/kubenrnetes/manifests/*.yml backups/
  3. run docker ps to make sure all statics pods are stopped.
  4. mv /var/lib/etcd /var/lib/etc-org
  5. now etcdctl snapshot restore …. --data-dir /var/lib/etcd
  6. once restore is is done, cp backup/*.yaml /etc/kubernets/manifests/
  7. wait for a minute for the static pods to startup, and everything should just work.