Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved scale-up detection. #647

Merged
merged 1 commit into from
Aug 2, 2023

Conversation

ishan16696
Copy link
Member

@ishan16696 ishan16696 commented Jul 20, 2023

/area high-availability
/kind technical-debt

What this PR does / why we need it:
While testing this PR: gardener/etcd-backup-restore#649 I found out that when third member of etcd cluster won't come up but pod spec has been updated, so etcd statefulset's status .status.updatedReplicas becomes 3:

status:
  availableReplicas: 2
  collisionCount: 0
  currentReplicas: 1
  currentRevision: etcd-main-79f88bd678
  observedGeneration: 2
  readyReplicas: 2
  replicas: 3
  updateRevision: etcd-main-86b48fdf9c
  updatedReplicas: 3

druid remove the scale-up annotation: gardener.cloud/scaled-to-multi-node from etcd statefulset as
func clusterScaledUpToMultiNode(){...} has returned false which is not correct.
more info please refer to this: #587 (comment), now it won't be an issue as backup-restore will skip checking scale-up conditions for first member

Which issue(s) this PR fixes:
Fixes # #587 (comment)

Special notes for your reviewer:

Release note:

None

@ishan16696 ishan16696 requested a review from a team as a code owner July 20, 2023 05:04
@gardener-robot gardener-robot added area/high-availability High availability related kind/technical-debt Something that is only solved on the surface, but requires more (re)work to be done properly needs/review Needs review size/xs Size of pull request is tiny (see gardener-robot robot/bots/size.py) labels Jul 20, 2023
@gardener-robot-ci-2 gardener-robot-ci-2 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jul 20, 2023
@gardener-robot-ci-1 gardener-robot-ci-1 added needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Jul 20, 2023
Copy link
Contributor

@shreyas-s-rao shreyas-s-rao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ishan16696 thanks for the PR. I feel this may not help solve the corner-case issue we see with scale-up today. Can you please address my comment? Thanks.

pkg/component/etcd/statefulset/statefulset.go Show resolved Hide resolved
@ishan16696
Copy link
Member Author

@unmarshall can you also take a look as I'm addressing your comments in this PR. Thanks

@ashwani2k ashwani2k added this to the v0.19.0 milestone Jul 31, 2023
Copy link
Contributor

@unmarshall unmarshall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@gardener-robot gardener-robot added reviewed/lgtm Has approval for merging and removed needs/changes Needs (more) changes needs/review Needs review labels Aug 2, 2023
@gardener-robot-ci-1 gardener-robot-ci-1 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Aug 2, 2023
@shreyas-s-rao
Copy link
Contributor

MoM of out-of-band call between @ishan16696 ,@unmarshall and myself:

Copy link
Contributor

@shreyas-s-rao shreyas-s-rao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@shreyas-s-rao shreyas-s-rao merged commit 93deafa into gardener:master Aug 2, 2023
1 check passed
@gardener-robot gardener-robot added the status/closed Issue is closed (either delivered or triaged) label Aug 2, 2023
@ishan16696 ishan16696 deleted the fix/scaleup-annotation branch August 8, 2023 06:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/high-availability High availability related kind/technical-debt Something that is only solved on the surface, but requires more (re)work to be done properly needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) reviewed/lgtm Has approval for merging reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) size/xs Size of pull request is tiny (see gardener-robot robot/bots/size.py) status/closed Issue is closed (either delivered or triaged)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants