feat: Add timezone to kube_cronjob_info / Make kube_cronjob_next_schedule_time timezone-aware

[mrueg]

What this PR does / why we need it:
Cronjob is not timezone aware.
How does this change affect the cardinality of KSM: (increases, decreases or does not change cardinality)
Adds one label to the kube_cronjob_info metric.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #2206

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mrueg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [mrueg]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[dgrisonnet]

wouldn't it be better in its own metric rather than in the info metric?

[mrueg]

There's a 1:1 relationship, why would you prefer a dedicated metric for it?

[mrueg]

@dgrisonnet any feedback here?

[dgrisonnet]

_info is not a very intuitive suffix which is why I don't really like these metrics where we just end up dumping a bunch of labels that aren't really related between one another.

I don't know what was the historical reasoning for putting labels into these metrics, but I can also see some dedicated metrics for values part of objects spec:

kube-state-metrics/internal/store/pod.go

Lines 1212 to 1231 in 569e820

	func createPodRestartPolicyFamilyGenerator() generator.FamilyGenerator {
	return *generator.NewFamilyGeneratorWithStability(
	"kube_pod_restart_policy",
	"Describes the restart policy in use by this pod.",
	metric.Gauge,
	basemetrics.STABLE,
	"",
	wrapPodFunc(func(p *v1.Pod) *metric.Family {
	return &metric.Family{
	Metrics: []*metric.Metric{
	{
	LabelKeys: []string{"type"},
	LabelValues: []string{string(p.Spec.RestartPolicy)},
	Value: float64(1),
	},
	},
	}
	}),
	)
	}

To me, info should be limited to the default labels. Any other information that we want to expose should be in its own dedicated metric. What is your opinion on that?

[mrueg]

I agree on having a separate metric for anything that does not have a 1:n relationship within the object. For 1:1 relationships, additional time series make it difficult to correlate between multiple labels vs. simply extracting the ones you need:
max(resource_info) by (label1, label2, label3).
Trying to do the same with multiple metrics can get very annoying and more error-prone.

To me, the _info metric should describe all single-value keys of the object that are static over its lifetime and non-numeric.

[mrueg]

I tried to define it here:
the _info metric should describe all single-value keys of the object that are static over its lifetime and non-numeric

[rexagod]

I meant a documentation/standard of sorts in the repository that enforces this. Not sure if there is one.

[mrueg]

I can add that in a follow-up PR.

[mrueg]

I'm currently working on a tool that will generate ideal metric names but it will take some time.

[rexagod]

I believe the tool's definition of an incorrect or correct metric representation for a nested object needs to weigh in the point Damien raised above.

Basically, are we in consensus on what is the standard approach for representing a nested value (since more granularity leads to more joins, but more abstractions lead to more cardinal label-sets (which is comparatively more harmful))?

[logicalhan]

/triage accepted
/assign @dgrisonnet

[mrueg]

@dgrisonnet @rexagod anything left to do to get this change in?

[der-eismann]

@dgrisonnet @rexagod Sorry for the ping, but this feature is pretty important to us since we can't set the timezone on AWS' Bottlerocket OS. Is there any way to accelerate the review process?