Hello all, while taking backup and restore for etcd can we use peer.crt & p . .

satya jaswanth yannamani:
Hello all,

while taking backup and restore for etcd can we use peer.crt & peer.key instead of server.crt and its key ? because in exam its asking to use tls peer.crt and peer.key
the pod running is with server.crt and server.key.

Alistair Mackay:
If an exam question tells you to use a certain set of certificates, then those are the ones you must use.
Ultimately, etcdctl will connect and the backup file will be produced, or it won’t

satya jaswanth yannamani:
Thanks @Alistair Mackay , so i should have backed up using peer.crt instead of server.crt because the question says so,right?

Alistair Mackay:
The questions are explicit. They won’t say do one thing when they mean another.
At the end of the day, etcdctl will either make a backup, or fail with an authentication error if the certs don’t work

satya jaswanth yannamani:
I understand, but if i describe etcd pod then the cert-file & key-file are using server.crt & server.key, which means the etcd pod is connected to the server.crt, i am confused at this point, what should i do if the pod is connected to server.crt and the question says to use peer.crt?

Alistair Mackay:
peer.crt is still issued by etcd’s CA, therefore is a valid client cert to connect to the server (along with peer.key).
This certificate is more usually issued for different etcd servers in a multi-node etcd cluster to communicate with each other.

If you so wanted (and you would never have to do this in an exam), you could create your own x509 CSR and private key, and issue your own certificate from etcd CA and use that with etcdctl.

Alistair Mackay:
Like this

Alistair Mackay:
Commands used to create my own cert and key as shown below…

# Options I want for my certificate
cat > my-etcd-cert.cnf <<EOF
[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
[req_distinguished_name]
[v3_req]
basicConstraints = critical, CA:FALSE
keyUsage = critical, nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth
EOF

# Generate a private key
openssl genrsa -out my-etcd-cert.key 2048

# Generate a signing request using the key and options file
openssl req -new -key my-etcd-cert.key \
  -subj "/CN=me-its-me/O=kubernetes" -out my-etcd-cert.csr -config my-etcd-cert.cnf

# Generate a new certificate from the signing request, signing it with etcd's CA.
openssl x509 -req -in my-etcd-cert.csr \
  -CA /etc/kubernetes/pki/etcd/ca.crt \
  -CAkey /etc/kubernetes/pki/etcd/ca.key \
  -CAcreateserial \
  -out my-etcd-cert.crt \
  -extensions v3_req \
  -extfile my-etcd-cert.cnf \
  -days 1000

Then I used them to make the backup, which passes validation of the question…

ETCDCTL_API=3 etcdctl snapshot save /opt/snapshot-pre-boot.db --cacert /etc/kubernetes/pki/etcd/ca.crt --cert my-etcd-cert.crt --key my-etcd-cert.key 

Snapshot saved at /opt/snapshot-pre-boot.db

Alistair Mackay:
Bottom line - if the certificate is signed by etcd’s CA cert and can be used for client authentication, it can be used with etcdctl.

Alistair Mackay:

satya jaswanth yannamani:
@Alistair Mackay Thank you so much for the clear explanation.

Hi! I took my CKA exam yesterday, I got the question about ssh into node, and perform back up and restore of etcd with peer.key and peer.crt. If I passed those in when backing up, I received “no such file or directory” error on the peer.crt/peer.key. Any idea what to do here? Can’t find any examples out there that does this either.

I also tried server.crt and server.key, but all I got was “permission denied”, so I suspect I have to authenticate with the peer flags somehow(?) These peer flags were also not present when checking the etcd pod’s commands. I do see in the docs, under Securing Communication that there are some peer flags for configuring etcd with secure peer comm., but these flags were unknown to the etcdctl tool

Any help appreciated!

What gave you “permission denied”? trying to access the cert file, or etcd saying that the certs were not sufficient to authenticate?

Either of the cert/key pairs in /etc/kubernetes/pki/etcd along with ca.crt should be sufficient to take a backup, as they both have key usage of “TLS Web Server Authentication, TLS Web Client Authentication” and are both signed by ca.crt
Only if they were expired they wouldn’t work, and that would also likely mean the cluster would be broken.

When I tried to pass --cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key in the etcdctl snapshot save cmd, I was met with “permission denied”. When I tried --cert=/etc/kubernetes/pki/etcd/peer.crt --key=/etc/kubernetes/pki/etcd/peer.key, I was met with “no such file or directory”. I did not actually go to /etc/kubernetes/pki/etcd to verify the file names were correct, which I should have. I am just wondering what I might have done wrong, as I should be able to use either cert/key pair

Did you also pass --cacert=/etc/kubernetes/pki/etcd/ca.crt ?
It is required to be able to authenticate the certificate presented to etcdctl by etcd server.

Please read this.

Yes I also passed the --cacert. I followed the docs on this. I believe I should be able to use sudo to bypass the “permission denied” when passing server.key/server.crt. I just find it weird that it did not find the paths for the peer key/cert pair, when the exam explicitly told to do so. I also submitted a ticket to LFC, they responded that all was in order on their end

I can’t comment on what is exactly in the exam - they may move things around to see if you are paying attention, but the usual state of affairs is if you are on the control plane node for the cluster then all the certificate files are present in /etc/kubernetes/pki/ectd and you should already be root - just like KK exercises.

hello, I have the same problem, I could not perform the etcdbackup operation in the exam for 2 times. However, I am always successful in the tests and trials I do. Please share if there is a solution.

1 Like

Hi team, I also faced the same error. O tried the peer.key and peer.crt and face error that key dont have permission to perform this task even I added the cart. Not sure what os happening here. Please advice of anyone have anything to shear.

When you says root access it mean we need to run su -i to be root? Or just ssh to controlplane node will work here ?