Hi team,
I would appreciate it if someone could help me resolve my issue. I have been stuck for 2 days, let me explain.
I am trying to expose my Prometheus and Grafana services via an Ingress with the NGINX controller on a local Kubernetes cluster, but I am experiencing an issue with redirection to the authentication pages.
Problem Summary
Redirection and 404 Error:
Grafana: The application expects /login but receives /grafana/login, resulting in a 404 error.
Prometheus: The application expects /graph but receives /prometheus/graph, resulting in a 404 error.
Behavior with Single Path Configuration:
When a single Ingress configuration is used with the path / (e.g., for Grafana), access works correctly, indicating that the issue is related to path prefix handling.
I assume that the backend pods here are grafana and prometheus, and if so, the path you are using for each in the ingress rules should not include “grafana/” and “prometheus/” respectively. Try removing those, and just pathType: Prefix, and see if you do better here. The annotations will strip off the first part of the path as seen by the ingress controller, which I think is what you want.
The backend pods are indeed grafana and prometheus, I had tried the pathType prefix, and the problem remains the same, in the logs of my ingress-controller I see that it manages to reach the pods, but prometheus and graphana redirect the request to an authentication page, and it is this redirection that causes the problem so I get the 404 error.
This error indicates in the logs that the path is incorrect (IP/grafana/ login & IP/prometheus/graph) the path should be IP/login & IP/graph.
The same thing happens with argocd, I’ve tested a nginx pod that doesn’t redirect to another authentication page and it works without a problem, so my ingress-controller is working correctly.
