Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add required service when serviceMonitor is enabled #1509

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

bendikp
Copy link
Contributor

@bendikp bendikp commented Sep 5, 2024

Currently when enabling serviceMonitor the service isn't automatically enabled, and you have to explicitly enable the service in order for the serviceMonitor to work.

I think it would be nice to add the service when you enable serviceMonitor. Especially since the serviceMonitor don't work when you don't have a service.

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign a7i for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Sep 5, 2024
@k8s-ci-robot
Copy link
Contributor

Hi @bendikp. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ingvagabund
Copy link
Contributor

Some users might deploy their own service monitor. Or, use a different mechanism for consuming services. I wonder whether the charts are expected to provide a "full" installation by default. Or, rather ask users to be more aware of what they do.

@bendikp
Copy link
Contributor Author

bendikp commented Sep 10, 2024

Choosing the abstraction of the charts isn't easy. I guess I'm just used to getting a Service when enabling serviceMonitor. Enabling both service and serviceMonitor seems a bit redundant since they are so tightly coupled.

Having a more functional value like metrics.enabled: true|false might be a solution.

@ingvagabund
Copy link
Contributor

ingvagabund commented Sep 10, 2024

That would cause strange cases like metrics.enabled=true && serviceMonitor.enabled=false.

I wonder if there's any precedence for cases like this. I.e. a higher level abstraction on top of the existing chart configuration.

@rouke-broersma
Copy link

That would cause strange cases like metrics.enabled=true && serviceMonitor.enabled=false.
I wonder if there's any precedence for cases like this. I.e. a higher level abstraction on top of the existing chart configuration.

If I look at a couple of the helm charts that I consume, they all do this in a slightly different way, however I think in general most charts enable the metrics service when metrics is enabled, and enable the service monitor when service monitor is also enabled.

Argocd: https://github.com/argoproj/argo-helm/blob/687392688fdbc28eb0813b083e3e63771d138c5a/charts/argo-cd/templates/argocd-application-controller/metrics.yaml#L1

Ingress-nginx: https://github.com/kubernetes/ingress-nginx/blob/1c2aecbf02898845b45cee3d0b493347b440995e/charts/ingress-nginx/templates/controller-service-metrics.yaml#L1

metrics.enabled=true && serviceMonitor.enabled=false is not a strange case, because you can use other tooling than prometheus to consume the metrics. In that case you have no need for a serviceMonitor, but you do want to enable the metrics endpoint and the metrics server.

However some charts do enable the service also when the service monitor is enabled:

Cillium: Cilium: https://github.com/cilium/cilium/blob/685790550b375380dd8fac71cc2a37427d78fc40/install/kubernetes/cilium/templates/cilium-agent/service.yaml#L3

In general the helm construct I see most often in the helm charts I consume is the following:

    metrics:
      enabled: true # Enabled metrics port and enables metrics service
      serviceMonitor:
        enabled: true # Enables ServiceMonitor that consumes metrics service

@ingvagabund
Copy link
Contributor

ingvagabund commented Sep 17, 2024

@rouke-broersma thank you for exploring the existing solutions. Yeah, this change requires to reshape the current charts "api". So when the metrics are enabled, the service monitor (enabled by default) needs to be explicitly disabled if its functionality is not needed.

@ingvagabund
Copy link
Contributor

@bendikp would you be interested in reshaping the current chart configuration structure?

@rouke-broersma
Copy link

@rouke-broersma thank you for exploring the existing solutions. Yeah, this change requires to reshape the current charts "api". So when the metrics are enabled, the service monitor (enabled by default) needs to be explicitly disabled if its functionality is not needed.

Just to clear, the most common case is that both metrics.enabled and metrics.serviceMonitor.enabled are false by default and that both need to be explicitly enabled for both to be deployed. It is not the common case that metrics.enabled implies metrics.serviceMonitor.enabled. To be most specific, the common condition is:

{{- if and .Values.metrics.enabled .Values.metrics.serviceMonitor.enabled -}}

While the ask in this PR is:

{{- if or .Values.metrics.enabled .Values.metrics.serviceMonitor.enabled -}}

Eg of the common case:

This enables metrics service and optionally metrics port (on container) but not serviceMonitor:

metrics:
  enabled: true

This does not enable anything:

metrics:
  enabled: false
  serviceMonitor:
    enabled: true

This enables all of metrics port (on container), metrics service and serviceMonitor:

metrics:
  enabled: true
  serviceMonitor:
    enabled: true

@a7i
Copy link
Contributor

a7i commented Sep 27, 2024

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants