Issue with deployed rke2 second Ingress-Nginx Controller inside a Kubernetes cluster running on bare-metal.

[Ganesh-hub-5]

What happened:

We have deployed two Nginx ingress controller in our cluster. The first ingress controller works fine but we are facing issue with second one. When accessing the service through second ingress controller (http://x3.abc.com/sample), we get This site can’t be reached x3.abc.com took too long to respond. error. We followed the same step as the first ingress controller including assigning different IP address using metallb and different ingress class in different namespace.

What you expected to happen:

For testing purpose we deployed a simple nginx image using standard yaml files. Everything is up and running, No error logs in pods, service mapped correct endpoints and ingress resource got external IP assigned by Second nginx ingress controller But when we try to access http://x3.abc.com/sample gives Site can't be reached error.

NGINX Ingress controller version:

NGINX Ingress controller
Release: v1.10.1-hardened1
Build: git-b48c66a2f
Repository: https://github.com/rancher/ingress-nginx
nginx version: nginx/1.25.3

Kubernetes version:

Client Version: v1.30.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.11+rke2r1
Environment:

• Cloud provider or hardware configuration: Metallb

• OS: Red Hat Enterprise Linux

• Kernel (e.g. uname -a): Linux

• RKE2 Version: v1.28.11

• Metallb Version: v0.14.5

• How was the ingress-nginx-controller installed:
  helm -n abcx3apps get values rke2-ingress-nginx-abcx3
  

• Current State of the controller:

  • kubectl describe ingressclasses
    Name: nginx-abcx3
    Labels: app.kubernetes.io/component=controller
    app.kubernetes.io/instance=rke2-ingress-nginx-abcx3
    app.kubernetes.io/managed-by=Helm
    app.kubernetes.io/name=rke2-ingress-nginx
    app.kubernetes.io/part-of=rke2-ingress-nginx
    app.kubernetes.io/version=1.10.1
    helm.sh/chart=rke2-ingress-nginx-4.10.101
    Annotations: meta.helm.sh/release-name: rke2-ingress-nginx-abcx3
    meta.helm.sh/release-namespace: abcx3apps
    Controller: k8s.io/ingress-nginx-abcx3
    Events:

  • kubectl -n <ingresscontrollernamespace> get all -A -o wide
    All pods are up and running

  • kubectl -n <ingresscontrollernamespace> describe svc <ingresscontrollerservicename>
    

• Current state of ingress object, if applicable:
  

• Others:
  

How to reproduce this issue:
To reproduce the issue we just need to change the ingress controller to second nginx ingress controller

As minimally and precisely as possible. Keep in mind we do not have access to your cluster or application.
Help up us (if possible) reproducing the issue using minikube or kind.

Install minikube/kind

• Minikube https://minikube.sigs.k8s.io/docs/start/
• Kind https://kind.sigs.k8s.io/docs/user/quick-start/

Install the ingress controller

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/baremetal/deploy.yaml

Install an application that will act as default backend (is just an echo app)

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/docs/examples/http-svc.yaml

Create an ingress (please add any additional annotation required)

echo "
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: foo-bar
annotations:
kubernetes.io/ingress.class: nginx
spec:
ingressClassName: nginx # omit this if you're on controller version below 1.0.0
rules:
- host: foo.bar
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: http-svc
port:
number: 80
" | kubectl apply -f -

make a request

POD_NAME=$(k get pods -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx -o NAME)
kubectl exec -it -n ingress-nginx $POD_NAME -- curl -H 'Host: foo.bar' localhost

--->

Anything else we need to know:

[k8s-ci-robot]

This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[longwuyuan]

• remove the screenshots
• look at the template of a new bug report
• edit this issue description and answer the questions asked in the template of a new bug report in this issue description
• make sure to format everything in markdown
• without above, there is nothing to analyze or reproduce
• follow these steps to make sure that both controllers are unique https://kubernetes.github.io/ingress-nginx/faq/#multiple-controller-in-one-cluster

/remove-kind bug
/ind support

[longwuyuan]

/kind suppport

[k8s-ci-robot]

@longwuyuan: The label(s) kind/suppport cannot be applied, because the repository doesn't have them.

In response to this:

/kind suppport

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[longwuyuan]

/kind support