Sarma Pasumarthi:
Hello all, I saw a lot of ETCD backup process posts here but none of them are conclusive. So, I went ahead and prepared some good steps (keeping exam console window in perspective).
Please suggest any changes if required. Thank you.
- Make sure you are on Master/Controlplane node
- Identify etcd pod by running the following command:
k get pods --all-namespaces
- Run the following command to describe etcd pod:
k -n kube-system describe pod <etcd-pod-name-here>
- From this output, identify and copy the following lines (under “Command:” section) and paste them in notepad of exam window:
a) --cert-file=<value>
b) --data-dir=<value>
c) --listen-client-urls=<value>
d) --key-file=<value>
e) --peer-trusted-ca-file=<value> - Now, at command prompt, issue this command to get help on “etcdctl snapshot save”. Notice the “-h” option
ETCDCTL_API=3 etcdctl snapshot save -h
- From help output, identify and copy (to notepad) the necessary tags that should be passed along:
a) --cacert=
b) --cert=
c) --endpoints=
d) --key= - Based on this information, build the “snapshot save” command in notepad. It should look something like this. You can get more info from kubernetes documentation tab:
ETCDCTL_API=3 etcdctl --endpoints <listen-client-urls> --cert=<cert-file> --cacert=<peer-trusted-ca-file> --key=<key-file> snapshot save <path-from-exam-question>
- Once you replace values, it looks like this:
ETCDCTL_API=3 etcdctl --endpoints <https://127.0.0.1:2379> --cert=/etc/kubernetes/pki/etcd/server.crt --cacert=/etc/kubernetes/pki/etcd/ca.crt --key=/etc/kubernetes/pki/etcd/server.key snapshot save /opt/snapshot-pre-boot.db
- Run this command and make sure the snapshot is taken in target folder
- Now, to restore, use the help screen “ETCDCTL_API=3 etcdctl snapshot restore -h”. From out of help screen, identify and add the following to save command you created above.
–initial-advertise-peer-urls=“http://127.0.0.1:2380”
–initial-cluster=“master=http://127.0.0.1:2380”
–initial-cluster-token=“etcd-cluster-1”
–data-dir=/var/lib/etcd-from-backup
So, here’s final command:
ETCDCTL_API=3 etcdctl --endpoints <https://127.0.0.1:2379> --cert=/etc/kubernetes/pki/etcd/server.crt --cacert=/etc/kubernetes/pki/etcd/ca.crt --key=/etc/kubernetes/pki/etcd/server.key --data-dir=/var/lib/etcd-from-backup --initial-advertise-peer-urls="<http://127.0.0.1:2380>" --initial-cluster="default=<http://127.0.0.1:2380>" --initial-cluster-token="etcd-cluster-1" snapshot restore /opt/snapshot-pre-boot.db
- Once done, navigate to –data-dir path and make sure a new director by name “member” is created
- Final task is to make changes in “/etc/kubernetes/manifests/etcd.yaml” file.
Under “command” section, change –data-dir value to the --data-dir value you used in above command (i.e /var/lib/etcd-from-backup)
Under “command” section, add a new entry: - --initial-cluster-token=etcd-cluster-1 (This item, you can find in snapshot command above. Just copy, paste as it is… You need to add a hyphen (-) though)
Under “volumes” section, in “hostPath”, change the directory path to “–data-dir” path you used in above command. (i.e /var/lib/etcd-from-backup)
Under “volumeMounts” section, change this path to “–data-dir” path as well (i.e /var/lib/etcd-from-backup) - Since this etcd pod is deployed as static pod, it should automatically identify the changes and restart
Get status of newly created pod using: watch "docker ps -a | grep etcd" (you will see etcd pod started a few seconds ago and running)
- Optionally, you can use this command to check: ETCDCTL_API=3 etcdctl member list
- Finally, execute any kubectl commands and make sure you are getting results.