And to create/restore backup I used 1. `ETCDCTL_API=3 etcdctl --endpoints=<https . . .

Kubernetes-Slack Discussions
#21

Mumshad Mannambeth:
have you given it sometime for the services to load maybe?

#22

Mumshad Mannambeth:
is the etcd service running?

#23

Mumshad Mannambeth:
what about the api server? is that running?

#24

Gennway:

controlplane $ docker ps
CONTAINER ID        IMAGE                            COMMAND                  CREATED             STATUS              PORTS               NAMES
a10af121f4fb        09d665d529d0                     "kube-controller-man…"   13 minutes ago      Up 13 minutes                           k8s_kube-controller-manager_kube-controller-manager-controlplane_kube-system_f6a9bf2865b2fe580f39f07ed872106b_4
a971a359c74d        1b74e93ece2f                     "kube-apiserver --ad…"   14 minutes ago      Up 14 minutes                           k8s_kube-apiserver_kube-apiserver-controlplane_kube-system_75088ddce920f6e21f067d88eb2a552a_1
9cb33970bc4e        d4ca8726196c                     "etcd --advertise-cl…"   17 minutes ago      Up 17 minutes                           k8s_etcd_etcd-controlplane_kube-system_96735226acf89ece93e32b21726b9de5_0
db9b16bf3eee        <http://k8s.gcr.io/pause:3.2|k8s.gcr.io/pause:3.2>             "/pause"                 18 minutes ago      Up 18 minutes                           k8s_POD_etcd-controlplane_kube-system_96735226acf89ece93e32b21726b9de5_0
08dbe0492c7f        cbdc8369d8b1                     "kube-scheduler --au…"   18 minutes ago      Up 18 minutes                           k8s_kube-scheduler_kube-scheduler-controlplane_kube-system_5146743ebb284c11f03dc85146799d8b_1
8771fccbdc43        <http://quay.io/kodekloudhub/k8:master|quay.io/kodekloudhub/k8:master>   "sleep 10000"            24 minutes ago      Up 24 minutes                           controlplane
2f3277fea1ef        <http://k8s.gcr.io/pause:3.2|k8s.gcr.io/pause:3.2>             "/pause"                 24 minutes ago      Up 24 minutes                           k8s_POD_kube-scheduler-controlplane_kube-system_5146743ebb284c11f03dc85146799d8b_0
d72223c7bc4b        <http://k8s.gcr.io/pause:3.2|k8s.gcr.io/pause:3.2>             "/pause"                 24 minutes ago      Up 24 minutes                           k8s_POD_kube-controller-manager-controlplane_kube-system_f6a9bf2865b2fe580f39f07ed872106b_0
66dd4b419ea4        <http://k8s.gcr.io/pause:3.2|k8s.gcr.io/pause:3.2>             "/pause"                 24 minutes ago      Up 24 minutes                           k8s_POD_kube-apiserver-controlplane_kube-system_75088ddce920f6e21f067d88eb2a552a_0
controlplane $ kubectl get pods -A
No resources found
#25

Gennway:
seems like all docker containers work fine

#26

Gennway:
i’ve done exactly same steps as I did in etcd backup/restore lab, but in the mock exam it doesnt work, idk

#27

Mumshad Mannambeth:
Can you check the logs of the etcd container?

#28

Mumshad Mannambeth:
And the api server container

#29

Mumshad Mannambeth:
using the docker logs command

#30

Gennway:
etcd :

#31

Gennway:

2021-07-02 11:58:19.267343 I | etcdmain: etcd Version: 3.4.9
2021-07-02 11:58:19.267376 I | etcdmain: Git SHA: 54ba95891
2021-07-02 11:58:19.267383 I | etcdmain: Go Version: go1.12.17
2021-07-02 11:58:19.267385 I | etcdmain: Go OS/Arch: linux/amd64
2021-07-02 11:58:19.267388 I | etcdmain: setting maximum number of CPUs to 2, total number of available CPUs is 2
2021-07-02 11:58:19.267450 N | etcdmain: the server is already initialized as member before, starting as etcd member...
[WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap' flag instead
2021-07-02 11:58:19.267483 I | embed: peerTLS: cert = /etc/kubernetes/pki/etcd/peer.crt, key = /etc/kubernetes/pki/etcd/peer.key, trusted-ca = /etc/kubernetes/pki/etcd/ca.crt, client-cert-auth = true, crl-file = 
2021-07-02 11:58:19.268205 I | embed: name = controlplane
2021-07-02 11:58:19.268213 I | embed: data dir = /var/lib/etcd
2021-07-02 11:58:19.268216 I | embed: member dir = /var/lib/etcd/member
2021-07-02 11:58:19.268219 I | embed: heartbeat = 100ms
2021-07-02 11:58:19.268221 I | embed: election = 1000ms
2021-07-02 11:58:19.268223 I | embed: snapshot count = 10000
2021-07-02 11:58:19.268229 I | embed: advertise client URLs = <https://172.17.0.10:2379>
2021-07-02 11:58:19.268232 I | embed: initial advertise peer URLs = <https://172.17.0.10:2380>
2021-07-02 11:58:19.268236 I | embed: initial cluster = 
2021-07-02 11:58:19.289436 I | etcdserver: restarting member f2872e5e711415ed in cluster 7f286a391c00da73 at commit index 3093
raft2021/07/02 11:58:19 INFO: f2872e5e711415ed switched to configuration voters=()
raft2021/07/02 11:58:19 INFO: f2872e5e711415ed became follower at term 2
raft2021/07/02 11:58:19 INFO: newRaft f2872e5e711415ed [peers: [], term: 2, commit: 3093, applied: 0, lastindex: 3093, lastterm: 2]
2021-07-02 11:58:19.291333 W | auth: simple token is not cryptographically signed
2021-07-02 11:58:19.292115 I | mvcc: restore compact to 1872
2021-07-02 11:58:19.296760 I | etcdserver: starting server... [version: 3.4.9, cluster version: to_be_decided]
raft2021/07/02 11:58:19 INFO: f2872e5e711415ed switched to configuration voters=(17475987862193444333)
2021-07-02 11:58:19.297607 I | etcdserver/membership: added member f2872e5e711415ed [<https://172.17.0.10:2380>] to cluster 7f286a391c00da73
2021-07-02 11:58:19.297788 N | etcdserver/membership: set the initial cluster version to 3.4
2021-07-02 11:58:19.297955 I | etcdserver/api: enabled capabilities for version 3.4
2021-07-02 11:58:19.299497 I | embed: ClientTLS: cert = /etc/kubernetes/pki/etcd/server.crt, key = /etc/kubernetes/pki/etcd/server.key, trusted-ca = /etc/kubernetes/pki/etcd/ca.crt, client-cert-auth = true, crl-file = 
2021-07-02 11:58:19.299724 I | embed: listening for metrics on <http://127.0.0.1:2381>
2021-07-02 11:58:19.300013 I | embed: listening for peers on 172.17.0.10:2380
raft2021/07/02 11:58:20 INFO: f2872e5e711415ed is starting a new election at term 2
raft2021/07/02 11:58:20 INFO: f2872e5e711415ed became candidate at term 3
raft2021/07/02 11:58:20 INFO: f2872e5e711415ed received MsgVoteResp from f2872e5e711415ed at term 3
raft2021/07/02 11:58:20 INFO: f2872e5e711415ed became leader at term 3
raft2021/07/02 11:58:20 INFO: raft.node: f2872e5e711415ed elected leader f2872e5e711415ed at term 3
2021-07-02 11:58:20.593598 I | etcdserver: published {Name:controlplane ClientURLs:[<https://172.17.0.10:2379>]} to cluster 7f286a391c00da73
2021-07-02 11:58:20.593979 I | embed: ready to serve client requests
2021-07-02 11:58:20.594316 I | embed: ready to serve client requests
2021-07-02 11:58:20.598206 I | embed: serving client requests on 127.0.0.1:2379
2021-07-02 11:58:20.858059 I | embed: serving client requests on 172.17.0.10:2379
2021-07-02 11:58:31.175035 I | etcdserver/api/etcdhttp: /health OK (status code 200)
#32

Gennway:
and api :
https://pastebin.com/AFKg0bEu (too many logs for slack)

#33

Gennway:
from what I understand, api cannot connect to etcd

#34

Gennway:
even if etcd health is ok

#35

Mumshad Mannambeth:
Ok can you restart the api server then?

#36

Mumshad Mannambeth:
just delete the API server container

#37

Mumshad Mannambeth:
and it should auto-restart

#38

Mumshad Mannambeth:
https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/practice-questions-answers/cluster-maintenance/backup-etcd/etcd-backup-and-restore.md#4-modify-etckubernetesmanifestsetcdyaml|https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/practice-questions[…]wers/cluster-maintenance/backup-etcd/etcd-backup-and-restore.md

#39

Mumshad Mannambeth:
On a side note, the question for the mock exam is only to take a backup isn’t it? not to restore.

#40

Gennway:
yeah it is to backup, but I though that I can make some extra task for myself