Kubernetes Network Policy Issue

Playgrounds
#1

Hi all!
I am playing with CKS simulator from killer.sh and found a strange issue with network policies.
There are 3 pods running in a namespace which must NOT be able to access the metadata server available at 192.168.100.21 IP.
The manifest file from the solution:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: metadata-deny
  namespace: metadata-access
spec:
  podSelector: {}
  policyTypes:
  - Egress
  egress:
  - to:
    - ipBlock:
        cidr: 0.0.0.0/0
        except:
        - 192.168.100.21/32

But the pods are still able to access the metadata server.
The cluster is running on kubernetes version 1.30.1 and Weave CNI.
P.S. I reproduced this on my home lab cluster running on Canal CNI and on KodeKloud playground running on Calico CNI.
Any idea what could be the issue? (edited)

#2

That looks like a correct policy for the specification. If That IP was accessible even with the netpol in place, then it suggests an issue with the CNI.

#3

Thank you for your answer. I am wondering how can I debug the issue inside the CNI?
I encountered this issue across 3 different clusters with different CNI: Canal, Calico and Weave.

#4

What you are describing is very similar to NetworkPolicy Metadata Protection | Killer Shell CKS | Killercoda (also by the people that make killer.sh). The only difference with this one is that it applies to some rather than all pods.

See how this is verified and ensure you are doing similar on your own clusters. More detail on your own setup would help.

#5

Thanks to good guy Rafael Ascensão from Killer.sh Slack I found why it is not working. Apparently the ipBlock directive is oriented to resources OUTSIDE the cluster. So using an IP from outside made it work properly. Thus according to above I am wondering how is it possible to implement network policies for resources living in the cluster or the nodes by IP? Is this even supported currently?

#6

It wouldn’t be because IPs within the cluster are always changing. The nature of clusters is that pods are disposable, and they will 99.9% of the time restart with a different IP. That’s why you use selectors.

Sorry I didn’t think to ask if your test scenario in your own clusters was using a pod to simulate the metadata resource.