CKS - PSP and kube-apiserver pod

[djvassil]

Playing with PSPs I noticed that upon creation of a PSP object denying privileged pods and activation of PodSecurityPolicy admission controller on the kube-apiserver.yaml file, the kube-apiserver pod is no longer listed as a pod in kube-system namespace.

It seems logical as kube-apiserver pod runs as privileged container, however, apart from not showing up in the list of pods, all queries are successful.

Could it be that there is an explanation for this behavior i am missing here?

Thanks

[djvassil]

ok, looking at the logs i see:

“Failed creating a mirror pod for” err=“pods "kube-apiserver-controlplane" is forbidden: PodSecurityPolicy: unable to admit pod: []” pod=“kube-system/kube-apiserver-controlplane”

Can you please let me know how we deal with this situation in the exam but more importantly in the real world, after enabling PSP Admission controller?

Thanks

[Tej-Singh-Rana]

Hi @djvassil ,
Can you please tell me the lab name?

[djvassil]

Hi @Tej-Singh-Rana . It is Lab PSP in CKS lecture.

Thanks

[Ayman]

Hello djvassil,

Is that issue with question 4?

[djvassil]

Well, after PodSecurityPolicy is added as an additional admission plugin in the kube-apiserver manifest, the kube-apiserver pod is no longer showing in the list of running pods, but the api server is actually responding to kubectl and all other pods are up.

Studying a bit more i found out this is expected behavior if there is no default psp and relevant RBAC config, allowing the creation of pods in the kube-system namespace.

I think it would be great if this point is clarified in the lecture.

[Ayman]

Kindly check the attached screenshot

[djvassil]

Yes, this is fine until you deploy the psp.
Then you get:

Also:

[Ayman]

Thanks for reporting! We will check it

[Carlos-Alvarado-Mart]

I got it running for Kubeadm clusters!

You have to get the user that your kubeadm is using to create your static pods and add it to the cluster role binding, but that is not enough, you also have to add an annotation about seccomp. I included on my example also the required stuff to allow the NGINX Ingress Controller pod.

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: default-allow-all
  annotations:
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: '*'
spec:
  privileged: true
  seLinux:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  runAsUser:
    rule: RunAsAny
  fsGroup:
    rule: RunAsAny
  volumes:
  - '*'
  hostNetwork: true
  allowedCapabilities:
  - NET_BIND_SERVICE
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: psp-allow-all-role
rules:
- apiGroups: ['policy']
  resources: ['podsecuritypolicies']
  verbs:     ['use']
  resourceNames:
  - default-allow-all
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: psp-allow-all-rolebinding
roleRef:
  kind: ClusterRole
  name: psp-allow-all-role
  apiGroup: rbac.authorization.k8s.io
subjects:
# Authorize all service accounts in a namespace (recommended):
- kind: Group
  apiGroup: rbac.authorization.k8s.io
  name: system:serviceaccounts:kube-system
- kind: Group
  apiGroup: rbac.authorization.k8s.io
  name: system:serviceaccounts:ingress-nginx
- kind: Group
  apiGroup: rbac.authorization.k8s.io
  name: system:serviceaccounts:lens-metrics
- kind: Group
  apiGroup: rbac.authorization.k8s.io
  name: system:serviceaccounts:default
- kind: User
  apiGroup: rbac.authorization.k8s.io
  name: system:node:master.calvarado04.com