How Ingress works, with respect to annotation nginx.ingress.kubernetes.io/rewrite-target:

rishabh.e.sharma · June 2, 2024, 9:23pm

Hi,

I am interested to understand how ingress really works with respect to paths

like /wear ------> wear-service
like /watch -----> watch-service

Now, in the ingress chapter it is mentioned that if we don’t add nginx.ingress.kubernetes.io/rewrite-target=/
Then /wear will point to wear-service/wear which is not the right path.

but in K8s, we will have only clusterIP with port to hit the actual pod. So is it mandatory to include this annotation always in ingress manifests?

Official doc:- Rewrite - Ingress-Nginx Controller

I checked the example

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/use-regex: “true”
nginx.ingress.kubernetes.io/rewrite-target: /$2
name: rewrite
namespace: default
spec:
ingressClassName: nginx
rules:

host: rewrite.bar.com
http:
paths:
- path: /something(/|$)(.*)
  pathType: ImplementationSpecific
  backend:
  service:
  name: http-svc
  port:
  number: 80

It will convert:-

rewrite.bar.com/something rewrites to rewrite.bar.com/
rewrite.bar.com/something/ rewrites to rewrite.bar.com/
rewrite.bar.com/something/new rewrites to rewrite.bar.com/new

but how redirection or rewrite of URL will help us as clusterIP services are always without path, i.e. with clusterIP:port

So rewrite.bar.com/something/newrewrites torewrite.bar.com/new
/new will be added to the actual cluster service i.e. http-svc:80/new as it should give nothing as this doesn’t exist at all.

rob_kodekloud · June 2, 2024, 9:42pm

I’m a little limited in testing your yaml, since it’s been corrupted by the forum software. To prevent this, you need to put it into

code blocks
   like
   this
        using the </> key to create an empty block

As for your general question: the service descriptor has no relationship at all to the path that the ingress resource uses. I believe that you can get the path passed to the service (assuming it knows what to do with it), but mostly that’s not how people use ingress resources.

As far as what the ingress-nginx rewrite annotations do, there are a bunch of examples in their docs.

rishabh.e.sharma · June 2, 2024, 10:34pm

Sorry for the yaml paste issue

My question is:- Why we need nginx.ingress.kubernetes.io/rewrite-target: / this annotation if we use simple concept of service and ingress.

i.e. path /watch -----> watch-svc
Ingress controller should be smart enough to see if path mentioned in URL is /watch, then I will redirect to clusterIP service watch-svc:port and then ultimately pod.

Why this annotation is mandatory ?

rob_kodekloud · June 2, 2024, 11:09pm

Don’t know; I’m not sure which cases require it. You might want to ask the ingress-nginx developers; they hang out on Kubernetes slack on the #ingress-nginx-users channel.

Alistair_KodeKloud · June 3, 2024, 4:55am

The rewrite target performs a URL rewrite on the incoming URL before passing it to the backend service. URL rewriting is a very common operation in web servers and they all support it. nginx, which provides the ingress controller is itself web server software.

So, in the case of the watch/wear example, we have two kubernetes services wear-svc and watch-svc. Both of these services serve their sites from path /, i.e. if you hit the services directly from a test pod using curl, you will curl on http://watch-svc/ and http://wear-svc/

If you are coming in through the ingress (I’ll call it example.com for now as I can’t remember what it is in the lab without looking), this is a single point of access, so we hit the ingress via http://example.com/watch and http://example.com/wear

Without rewriting, then the URL paths would be passed unchanged to the two backend services, i.e. http://watch-svc/watch and http://wear-svc/wear, both which would result in a 404.

The rewrite target on each ingress tells nginx in the ingress controller to create rewrite rules to rewrite the URL path to / so it will then route

http://example.com/watch → http://watch-svc/
http://example.com/wear → http://wear-svc/

You can do much more complex rewrites depending on the requirements of the applications and backend services.

rishabh.e.sharma · June 3, 2024, 6:14am

Thanks

@Alistair_KodeKloud May I ask why the path will be added in front of service DNS name ?
If we mentioned, if path observed is /wear then just forward to wear-svc at port 80.

This is bit confusing, Also let me show you one more observation:-

Furthermore, Let’s look at this example :-

Here we have two services
asia
europe

and we make one ingress resource world

Now the I made one ingress without this annotation as follows:- All commands with working ingress without this above annotation.

AME READY STATUS RESTARTS AGE
asia-5b7655fc9d-2hpgg 1/1 Running 0 4m44s
asia-5b7655fc9d-p2xps 1/1 Running 0 4m44s
europe-7bbdc46d74-kn5fp 1/1 Running 0 4m44s
europe-7bbdc46d74-vchz7 1/1 Running 0 4m44s
controlplane $ k get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
asia ClusterIP 10.101.196.128 80/TCP 4m5s
europe ClusterIP 10.108.159.252 80/TCP 3m59s
controlplane $ k create ing world --rule=world.universe.mine/europe=europe:80 --rule=world.universe.mine/asia=asia:80
ingress.networking.k8s.io/world created
controlplane $ k get svc -n ingress-nginx | grep NodePort
ingress-nginx-controller NodePort 10.109.206.8 80:30080/TCP,443:30443/TCP 8m25s
controlplane $
controlplane $
controlplane $
controlplane $ curl world.universe.mine:30080/asia/
hello, you reached ASIA
controlplane $ curl world.universe.mine:30080/europe/
hello, you reached EUROPE
controlplane $

Do you know why it’s working without annotation ?
Of course, if we add the annotation then also it works fine.

It is written in killer.sh

   # this annotation removes the need for a trailing slash when calling urls
    # but it is not necessary for solving this scenario
    nginx.ingress.kubernetes.io/rewrite-target: /

Alistair_KodeKloud · June 3, 2024, 7:10pm

Let’s look at what happens with and without the annotation when you access the path without a trailing slash. For this we use -I flag of curl

With annotation

controlplane $ curl http://world.universe.mine:30080/europe 
hello, you reached EUROPE

controlplane $ curl -I  http://world.universe.mine:30080/europe
HTTP/1.1 200 OK
Date: Mon, 03 Jun 2024 18:57:41 GMT
Content-Type: text/html
Content-Length: 26
Connection: keep-alive
Last-Modified: Mon, 03 Jun 2024 18:53:50 GMT
ETag: "665e113e-1a"
Accept-Ranges: bytes

The text is retuned correctly.

Without annotation

controlplane $ curl  http://world.universe.mine:30080/europe
<html>
<head><title>301 Moved Permanently</title></head>
<body>
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx/1.21.5</center>
</body>
</html>

…and the headers

controlplane $ curl -I  http://world.universe.mine:30080/europe
HTTP/1.1 301 Moved Permanently
Date: Mon, 03 Jun 2024 18:59:36 GMT
Content-Type: text/html
Content-Length: 169
Connection: keep-alive
Location: http://world.universe.mine/europe/

The redirect is to a version of the URL with a trailing slash. For curl to resolve that correctly without adding the slash yourself, you have to tell it to follow redirects, however there’s a small issue here in that the redirect is unaware of the nodeport number, so assumes port 80 because it’s HTTP, thus getting curl to follow the redirect fails.

controlplane $ curl -L  http://world.universe.mine:30080/europe
curl: (7) Failed to connect to world.universe.mine port 80: Connection refused

rishabh.e.sharma · June 3, 2024, 8:33pm

Thanks @Alistair_KodeKloud for the explanation

I would like to know, it was working with trailing slash, right? (without annotation)

ie. controlplane $ curl world.universe.mine:30080/europe/
hello, you reached EUROPE

As per the rule without annotation the target should be like this.
world.universe.mine:30080/europe/. ---------> europe:80/europe
which shouldn’t work but it is working,

I just want to understand the functioning of the annotation, most of the time we are using it.

one of the reason is that you explained previously that otherwise the path will be added in front of the svc itself
i.e. europe:80/europe
other reason is trailing slash in from the URL that we use to hit the ingress-controller.

Alistair_KodeKloud · June 4, 2024, 8:36pm

To really see what goes on takes an understanding of some rather complex nginx configuration. Whenever an Ingress is deployed or edited, the configuration in the nginx controller is updated.

The ingress controller is nothing more than nginx server running in reverse proxy mode, which means it can examine incoming requests and make a routing decision (which backend to use) based on a set of rules for matching URL paths (and a lot of other stuff as well using other annotations).

To see the differences in the Killercoda lab

Deploy the solution as given - don’t end the lab

Get the nginx configuration and save as rewrite.conf

controller_pod=$(kubectl get pod -n ingress-nginx --selector app.kubernetes.io/component=controller)
kubectl exec -t -n ingress-nginx $controller_pod -- cat nginx.conf > rewrite.conf

Edit the ingress and remove the annotation. Wait 30 seconds to ensure controller has updated

Get the configuration again and save as no-rewrite.conf

kubectl exec -t -n ingress-nginx $controller_pod -- cat nginx.conf > no-rewrite.conf

Use the VSCode editor by pressing Editor tab at the top left of the terminal
Right click on rewrite.conf as shown and hit Select for Compare

image700×871 42.5 KB
Right click on no-rewrite.conf as shown and hit Compare with Selected

The editor will show the two configurations side by side with differences highlighted. Lines in red on the left were removed from the config when you updated the ingress to remove the annotation, and stuff on the right in green was added to the config

You will see one of the things removed for each of europe and asia is the rewrite rule created by the annotation, e.g.

rewrite "(?i)/europe" / break;

rewrite directive: This is used to rewrite URLs based on specified patterns and conditions.
(?i): This is a flag within the regular expression that makes the pattern case-insensitive. Thus, it will match /europe, /Europe, /EUROPE, etc. (although technically it will only match the lowercase version due to the location directive explained below)
/europe: This is the pattern that nginx will look for in the requested URL.
/: This is the replacement string. When the pattern is matched, the URL is rewritten to just /.
break: This indicates that nginx should stop processing the rewrite directives at this point and immediately process the rewritten URL.

Putting it all together, this rewrite rule means that any URL containing /europe (case-insensitive) will be rewritten to /, and nginx will stop any further rewrite processing for this request. So the backend service will be requested as europe.world.service/

That was part of a single location processing block:

location ~* "^/europe" {

which matches any URL path starting with /europe (exact case) and anything after it e.g. /europe/london, /europe/London but not /Europe, /EUROPE

In the replacement without the rewrite, two distinct exact matching location rules have been added, one for /europe and another for /europe/. Here, ultimately the former will generate a “301 Moved Permanently” response pointing to the latter, and the latter will explicitly call the backend on /

All the above is also done for asia