fix: use dl.k8s.io, not kubernetes-release bucket #2872

rjsadow · 2023-05-12T01:19:58Z

This commit improves the URL handling for resource extraction and version filtering processes. The following changes were made:

In the kinder/pkg/extract/extract.go file:

The releaseBuildURepository constant was updated to use the new "https://dl.k8s.io/release" URL for release builds. In the tests/e2e/manifests/verify_manifest_lists.go file:

The URL used in the filterVersions() function was updated to use the new "https://dl.k8s.io/release" URL for retrieving stable versions.

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

There are still references to https://storage.googleapis.com/kubernetes-release instead of https://dl.k8s.io/

dl.k8s.io is the correct advertised download host and will eventually move to be fastly shielding a fully community-owned bucket

ref: kubernetes/k8s.io#2396

This commit improves the URL handling for resource extraction and version filtering processes. The following changes were made: In the kinder/pkg/extract/extract.go file: The releaseBuildURepository constant was updated to use the new "https://dl.k8s.io/release" URL for release builds. In the tests/e2e/manifests/verify_manifest_lists.go file: The URL used in the filterVersions() function was updated to use the new "https://dl.k8s.io/release" URL for retrieving stable versions. Signed-off-by: Ricky Sadowski <[email protected]>

neolit123

There are still references to https://storage.googleapis.com/kubernetes-release instead of https://dl.k8s.io/

is it possible to change all k/kubeadm references in this PR?

neolit123

There are still references to https://storage.googleapis.com/kubernetes-release instead of https://dl.k8s.io/

is it possible to change all k/kubeadm references in this PR?

rjsadow · 2023-05-12T10:33:34Z

There are still references to https://storage.googleapis.com/kubernetes-release instead of https://dl.k8s.io/

is it possible to change all k/kubeadm references in this PR?

Yes! Totally, can you help me understand what other references you're seeing? I believe this PR already covers changing all the old references. There are still some instances to gs://kubernetes-release, but I intend on handling those changes in a different PR.

neolit123 · 2023-05-12T12:23:57Z

There are still some instances to gs://kubernetes-release, but I intend on handling those changes in a different PR.

you can also add them here in a separate commit

rjsadow · 2023-05-12T12:53:13Z

Updated with the gs:// doc references.

docs/testing-pre-releases.md

tests/e2e/manifests/verify_manifest_lists.go

Signed-off-by: rsadowsk <[email protected]>

rjsadow · 2023-05-13T18:50:50Z

@neolit123, I reworked the ### Getting kubeadm binaries from a release bucket a bit. Please take a look and let me know if you're ok with it. Sig-k8s-io is discouraging folks from browsing the release buckets. So instead this change points them to the release page to search for any versions they're interested in.

pacoxu

/lgtm

neolit123 · 2023-05-15T09:32:33Z

docs/testing-pre-releases.md

-| **GA release**          | from .deb or .rpm repository | from [github release page](https://github.com/kubernetes/kubernetes/releases) or from `gs://kubernetes-release/release/` GCS bucket | from `k8s.gcr.io` container registry or from [github release page](https://github.com/kubernetes/kubernetes/releases) |
-| **alpha/beta release*** | not available.               | from [github release page](https://github.com/kubernetes/kubernetes/releases) or from `gs://kubernetes-release/release/` GCS bucket | from `k8s.gcr.io` container registry or from [github release page](https://github.com/kubernetes/kubernetes/releases) |
+| **GA release**          | from .deb or .rpm repository | from [github release page](https://github.com/kubernetes/kubernetes/releases) or from `https://dl.k8s.io/` release bucket | from `k8s.gcr.io` container registry or from [github release page](https://github.com/kubernetes/kubernetes/releases) |
+| **alpha/beta release*** | not available.               | from [github release page](https://github.com/kubernetes/kubernetes/releases) or from `https://dl.k8s.io/` release bucket | from `k8s.gcr.io` container registry or from [github release page](https://github.com/kubernetes/kubernetes/releases) |
 | **CI/CD release***      | not available.               | from `gs://k8s-release-dev/ci/` GCS bucket (built every merge)                                                                      | from `gcr.io/k8s-staging-ci-images` container registry (built every few hours, not by PR)                             |


what is the plan for
gs://k8s-release-dev/ci/, can we change it with its pot. replacement in this PR?

Asking kubernetes/k8s.io#2396 (comment), rework based on the sig's recommendation.

k8s-release-dev will be tracked separately and isn't meant to be end-user facing (unlike dl.k8s.io), it's meant for contributors to the project

kubernetes/k8s.io#2396 (comment)

neolit123 · 2023-05-15T09:33:23Z

docs/testing-pre-releases.md


-Pre-compiled GA, alpha/beta versions of kubeadm binary are deployed into `gs://kubernetes-release/release/` GCS bucket,
+Pre-compiled GA, alpha/beta versions of kubeadm binary are deployed into `https://dl.k8s.io/release/` release bucket,


@pacoxu mentioned this URL is a 404.

It should be a 302 if done with curl. Through the reverse proxy you dan't browse directories which may be why you're getting a 404 but if you look at a specific file, it should be there.

$ curl -s -o /dev/null -w "%{http_code}\n" https://dl.k8s.io/release/ 302 $ curl -sL -o /dev/null -w "%{http_code}\n" https://dl.k8s.io/release/ 404 $ curl -sL -o /dev/null -w "%{http_code}\n" https://dl.k8s.io/release/stable.txt 200

sgtm,

@pacoxu please let us know if you see a problem with the doc text, as is

neolit123 · 2023-05-15T09:33:46Z

docs/testing-pre-releases.md


-Pre-compiled GA, alpha/beta versions of kubeadm binary are deployed into `gs://kubernetes-release/release/` GCS bucket,
+Pre-compiled GA, alpha/beta versions of kubeadm binary are deployed into `https://dl.k8s.io/release/` release bucket,
 while CI/CD versions are deployed into `gs://k8s-release-dev/ci/` bucket.


same Q as above for the CI bucket

k8s-release-dev will be tracked separately and isn't meant to be end-user facing (unlike dl.k8s.io), it's meant for contributors to the project

kubernetes/k8s.io#2396 (comment)

docs/testing-pre-releases.md

neolit123 · 2023-05-15T09:37:25Z

docs/testing-pre-releases.md

@@ -167,8 +154,8 @@ Run `cluster/get-kube-binaries.sh` to download the tarball with server binaries.

 > Inside release notes, usually there is a direct link for getting server binaries directly

-> `cluster/get-kube-binaries.sh` retrieves server binaries from `gs://kubernetes-release/release/{release}`
-  GCS bucket; you can use `gsutil` to get server binaries directly.
+> `cluster/get-kube-binaries.sh` retrieves server binaries from `https://dl.k8s.io/release/{release}`


note, in the future we should clean all /cluster/x referenced

Were those fully depricated or were they moved somewhere else?

i haven't checked. we should not ref /cluster tools from k/kubeadm, since /cluster should not be used directly by users. instead users should curl, perhaps.

Co-authored-by: Lubomir I. Ivanov <[email protected]>

neolit123 · 2023-05-15T13:31:04Z

so since the scope of the PR is to change release bucket refs we could merge it early, but it means we need to log a k/kubeadm issue with remaining work.

e.g.

remove / change all gs:// entries in docs
remove /cluster/* script examples in docs
other?

rjsadow · 2023-05-15T15:18:41Z

remove / change all gs:// entries in docs

Given the advice from sig-k8s-infra, I think a separate issue would be appropriate. The transition will be a phased effort so I think it's premature to attempt to change all references at once.

neolit123 · 2023-05-15T15:23:23Z

remove / change all gs:// entries in docs

Given the advice from sig-k8s-infra, I think a separate issue would be appropriate. The transition will be a phased effort so I think it's premature to attempt to change all references at once.

ack, i will log the issue

rjsadow · 2023-05-15T15:27:40Z

ack, i will log the issue

I opened #2875, feel free to close.

neolit123 · 2023-05-15T15:33:20Z

ack, i will log the issue

I opened #2875, feel free to close.

logged
#2876
thanks

neolit123

/lgtm
/approve
/hold

@pacoxu PTAL and remove the hold if this seems ok to you

k8s-ci-robot · 2023-05-16T13:22:11Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: neolit123, pacoxu, rjsadow

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [neolit123]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

pacoxu · 2023-05-16T13:22:29Z

/unhold
thanks

k8s-ci-robot added kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels May 12, 2023

k8s-ci-robot requested a review from fabriziopandini May 12, 2023 01:20

k8s-ci-robot added the area/kinder Issues to track work in the kinder tool label May 12, 2023

k8s-ci-robot requested a review from pacoxu May 12, 2023 01:20

k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label May 12, 2023

rjsadow mentioned this pull request May 12, 2023

Deprecate and migrate away from gs://kubernetes-release kubernetes/k8s.io#2396

Open

neolit123 reviewed May 12, 2023

View reviewed changes

neolit123 added this to the v1.28 milestone May 12, 2023

k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels May 12, 2023

neolit123 reviewed May 12, 2023

View reviewed changes

docs/testing-pre-releases.md Outdated Show resolved Hide resolved

neolit123 reviewed May 12, 2023

View reviewed changes

tests/e2e/manifests/verify_manifest_lists.go Show resolved Hide resolved

neolit123 mentioned this pull request May 12, 2023

fix broken manifest list e2e due to arm arch #2873

Closed

rjsadow force-pushed the main branch 3 times, most recently from b10028b to 3aa32d8 Compare May 13, 2023 18:46

doc: update references from gs://kubernetes-release to https://dl.k8s.io

7da4109

Signed-off-by: rsadowsk <[email protected]>

rjsadow force-pushed the main branch from 3aa32d8 to 7da4109 Compare May 13, 2023 18:48

pacoxu reviewed May 15, 2023

View reviewed changes

k8s-ci-robot assigned pacoxu May 15, 2023

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 15, 2023

neolit123 reviewed May 15, 2023

View reviewed changes

docs/testing-pre-releases.md Outdated Show resolved Hide resolved

neolit123 reviewed May 15, 2023

View reviewed changes

Apply suggestions from code review

7891d05

Co-authored-by: Lubomir I. Ivanov <[email protected]>

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 15, 2023

neolit123 mentioned this pull request May 15, 2023

cleanup docs post GCS infra migration #2876

Open

3 tasks

neolit123 reviewed May 15, 2023

View reviewed changes

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 15, 2023

k8s-ci-robot assigned neolit123 May 15, 2023

k8s-ci-robot added lgtm "Looks good to me", indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels May 15, 2023

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 16, 2023

k8s-ci-robot merged commit 7ca96ba into kubernetes:main May 16, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: use dl.k8s.io, not kubernetes-release bucket #2872

fix: use dl.k8s.io, not kubernetes-release bucket #2872

rjsadow commented May 12, 2023

neolit123 left a comment

neolit123 left a comment

rjsadow commented May 12, 2023 •

edited

Loading

neolit123 commented May 12, 2023

rjsadow commented May 12, 2023

rjsadow commented May 13, 2023

pacoxu left a comment

neolit123 May 15, 2023

rjsadow May 15, 2023

rjsadow May 15, 2023

neolit123 May 15, 2023 •

edited

Loading

neolit123 May 15, 2023

rjsadow May 15, 2023 •

edited

Loading

neolit123 May 15, 2023

pacoxu May 16, 2023

neolit123 May 15, 2023

rjsadow May 15, 2023

neolit123 May 15, 2023

rjsadow May 15, 2023

neolit123 May 15, 2023

neolit123 commented May 15, 2023

rjsadow commented May 15, 2023

neolit123 commented May 15, 2023

rjsadow commented May 15, 2023

neolit123 commented May 15, 2023

neolit123 left a comment

k8s-ci-robot commented May 16, 2023

pacoxu commented May 16, 2023


		Pre-compiled GA, alpha/beta versions of kubeadm binary are deployed into `gs://kubernetes-release/release/` GCS bucket,
		Pre-compiled GA, alpha/beta versions of kubeadm binary are deployed into `https://dl.k8s.io/release/` release bucket,

fix: use dl.k8s.io, not kubernetes-release bucket #2872

fix: use dl.k8s.io, not kubernetes-release bucket #2872

Conversation

rjsadow commented May 12, 2023

What type of PR is this?

What this PR does / why we need it:

neolit123 left a comment

Choose a reason for hiding this comment

neolit123 left a comment

Choose a reason for hiding this comment

rjsadow commented May 12, 2023 • edited Loading

neolit123 commented May 12, 2023

rjsadow commented May 12, 2023

rjsadow commented May 13, 2023

pacoxu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

neolit123 May 15, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rjsadow May 15, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

neolit123 commented May 15, 2023

rjsadow commented May 15, 2023

neolit123 commented May 15, 2023

rjsadow commented May 15, 2023

neolit123 commented May 15, 2023

neolit123 left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented May 16, 2023

pacoxu commented May 16, 2023

rjsadow commented May 12, 2023 •

edited

Loading

neolit123 May 15, 2023 •

edited

Loading

rjsadow May 15, 2023 •

edited

Loading