Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regression in behavior when setting load balancer source ranges for a Service #6492

Open
elmiko opened this issue Jun 27, 2024 · 6 comments
Open
Labels
kind/bug Categorizes issue or PR as related to a bug.

Comments

@elmiko
Copy link

elmiko commented Jun 27, 2024

What happened:

When setting both .spec.loadBalancerSourceRanges and the service.beta.kubernetes.io/load-balancer-source-ranges annotation on a Service object, the CCM will not reconcile the Service nor will it allow for its deletion. The following error is present:

E0612 16:22:37.193349       1 azure_loadbalancer.go:2952] "Failed to parse access control configuration for service" err="cannot set both spec.LoadBalancerSourceRanges and service annotation service.beta.kubernetes.io/azure-allowed-ip-ranges" logger="reconcileSecurityGroup" cluster="ci-op-mv04309s-04a70-6jszn" service="openshift-ingress/router-sourcerangesstatus" load-balancer="ci-op-mv04309s-04a70-6jszn" delete-lb=true

What you expected to happen:

Expected the Service to continue to be reconciled even if one of the source ranges is ignored.

How to reproduce it (as minimally and precisely as possible):

  1. Create a service with both service.beta.kubernetes.io/load-balancer-source-ranges annotation and spec.LoadBalancerSourceRanges

Anything else we need to know?:

We (Red Hat) discovered this regression in our automated testing, and while the user behavior of setting both ranges may not be supported by the CCM, the error when reconciling makes it a possible upgrade blocker for users who may have (inadvertently or not) set both these range options.

There are 2 pull requests which appear to have combined to create this regression:

the effect of #5164 resulted in the following logic:

# If result is false then either not valid or no source ranges provided:
(spec.LoadBalancerSourceRanges OR service.beta.kubernetes.io/load-balancer-source-ranges) XOR (service.beta.kubernetes.io/azure-allowed-ip-ranges)

We do not have automation around the service.beta.kubernetes.io/azure-allowed-ip-ranges annotation, and as such we have not directly noticed this failure but we believe it exists.

when combined with #5885, the resulting logic becomes:

# If result is false then either not valid or no source ranges provided:
(spec.LoadBalancerSourceRanges) XOR (service.beta.kubernetes.io/load-balancer-source-ranges OR service.beta.kubernetes.io/azure-allowed-ip-ranges)

Related report in Red Hat bug tracking OCPBUGS-35757.

Environment:

This was tested on the upcoming release of OpenShift which uses Kubernetes version 1.30 and RHCOS 9.4 instances.

@elmiko elmiko added the kind/bug Categorizes issue or PR as related to a bug. label Jun 27, 2024
@elmiko
Copy link
Author

elmiko commented Jun 27, 2024

cc @JoelSpeed @gcs278

@elmiko
Copy link
Author

elmiko commented Jun 27, 2024

one thing that came up in discussion internally was whether we missed an announcement or release note about this change in behavior. is this just an organic regression or did we miss something about it?

@JoelSpeed
Copy link
Contributor

@feiskyer @bridgetkromhout Can you please review this issue?

@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Oct 28, 2024
@JoelSpeed
Copy link
Contributor

/remove-lifecycle stale

This is still an issue

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Oct 28, 2024
@nilo19
Copy link
Contributor

nilo19 commented Oct 31, 2024

@zarvd could you help one this one? Do you think we should ignore spec.LoadBalancerSourceRanges if the annotation counterpart is set to support a smooth migration?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

No branches or pull requests

5 participants