Tasks to bump to Kubernetes v1.31

[sbueringer]

This issue is tracking the tasks that should be implemented after the Kubernetes minor release has been released.

Tasks

Note: If feasible we usually cherry-pick the changes back to the latest release series.

Supporting managing and running on the new Kubernetes version (@sbueringer / @chrischdi)

This section contains tasks to update our book, e2e testing and CI to use and test the new Kubernetes version
as well as changes to Cluster API that we might have to make to support the new Kubernetes version. All of these
changes should be cherry-picked to all release series that will support the new Kubernetes version.

• Continuously modify CAPD to use early versions of the upcoming Kubernetes release (betas and rcs):

  • Bump the Kubernetes version in test/* except for test/infrastructure/kind/*.
  • Prior art: ✨ Bump Kubernetes version used for testing to v1.30.0-rc.1 #10384
  • ✨ Bump Kubernetes version used for testing to v1.31.0-beta.0 #10922
  • ✨ Bump Kubernetes version used for testing to v1.31.0-rc.0 #10965
  • ✨ Bump Kubernetes to v1.31.0-rc.1 #11021

• Modify CAPD to use the new Kubernetes release after it is GA:

  • Bump the Kubernetes version in test/* except for test/infrastructure/kind/*.
  • Prior art: 🌱 Bump Kubernetes in tests to v1.30.0 and claim support for v1.30 #10454
  • PR: ✨ Bump Kubernetes in tests to v1.31.0 and claim support for v1.31 #11030

• Ensure the jobs are adjusted to provide test coverage according to our support policy:

  PR: CAPI: adjust jobs due to v1.31 GA kubernetes/test-infra#33294

  • At the .versions section in the cluster-api-prowjob-gen.yaml file in test-infra:

    • Add a new entry for the new Kubernetes version
    • Adjust the released Kubernetes's version entry to refer stable-1.<minor> instead of ci/latest-1.<minor>
    • Check and update the versions for the keys etcd and coreDNS if necessary:
      • For etcd, see the DefaultEtcdVersion kubeadm constant: e.g. for v1.28.0
      • For coredns, see the CoreDNSVersion kubeadm constant:e.g. for v1.28.0

  • For the .branches.main section in the cluster-api-prowjob-gen.yaml file in test-infra:

    • For the .upgrades section:
      • Drop the oldest upgrade
      • Add a new upgrade entry from the previous to the new Kubernetes version
    • Bump the version set at .kubernetesVersionManagement to the new minimum supported management cluster version (This is the image version available as kind image).
    • Bump the version set at .kubebuilderEnvtestKubernetesVersion to the new minimum supported management cluster version.

  • Run make generate-test-infra-prowjobs to generate the resulting prowjob configuration:

    TEST_INFRA_DIR=../../k8s.io/test-infra make generate-test-infra-prowjobs

  • Prior art: CAPI: adjust jobs due to v1.30 GA kubernetes/test-infra#32456

• Update book:

  • Update supported versions in versions.md
  • Prior art: 🌱 Bump Kubernetes in tests to v1.30.0 and claim support for v1.30 #10454
  • PR: ✨ Bump Kubernetes in tests to v1.31.0 and claim support for v1.31 #11030

• Issues specific to the Kubernetes minor release:

  • Sometimes there are adjustments that we have to make in Cluster API to be able to support
    a new Kubernetes minor version. Please add these issues here when they are identified.
  • Consider
    • Implement support for kubeadm v1beta4 API #10708
      • PR: ✨ Add kubeadm v1beta4 types #10709
      • PR: [release-1.7] ✨ Add kubeadm v1beta4 types #10771
    • Handle kube-proxy v1alpha2 migration #10737

Bump quickstart and kind image references in CAPD

Prerequisites:

• The target Kubernetes version is GA

• There is a new kind version with/or a new set of kind images for the target Kubernetes version

• Bump quickstart and kind image references in CAPD:

  • Bump the Kubernetes version in:
    • docs/*
    • Tiltfile
  • Bump kind image references in CAPD (and also kind if necessary, including the latest images for this kind release)
    • Add new images in the kind mapper.go.
      • See the kind releases page for the list of released images.
    • Set new default image for the test framework
    • If code changes are required for CAPD to incorporate the new Kind version, update kind latestMode
  • Verify the quickstart manually
  • Prior art: 🌱 Bump kind to v0.23.0 #10610
  • PR: 🌱 Bump kind to v0.24.0 #11081

• Cherry-pick above PR to the latest release branch.

  • PR: 🌱 [release-1.8] Bump kind to v0.24.0 #11084

Using new Kubernetes dependencies

This section contains tasks to update Cluster API to use the latest Kubernetes Go dependencies and related topics
like using the right Go version and build images. These changes are only made on the main branch. We don't
need them in older releases as they are not necessary to manage workload clusters of the new Kubernetes version or
run the Cluster API controllers on the new Kubernetes version.

• Ensure there is a new controller-runtime minor release which uses the new Kubernetes Go dependencies.
• Update our ProwJobs for the main branch to use the kubekins-e2e with the correct Kubernetes version via cluster-api-prowjob-gen.yaml and by running make generate-test-infra-prowjobs.
  • It is recommended to have one PR for presubmit and one for periodic jobs to reduce the risk of breaking the periodic jobs.
  • Prior art: cluster-api: Bump kubekins image for main to 1.30 to use go 1.22 kubernetes/test-infra#32380
  • PR: cluster-api: Bump kubekins to 1.31 for main kubernetes/test-infra#33367
• Bump the Go version in Cluster API: (if Kubernetes is using a new Go minor version) [does not apply, still Go 1.22]
  • Search for the currently used Go version across the repository and update it
  • We have to at least modify it in: hack/ensure-go.sh, .golangci.yml, cloudbuild*.yaml, go.mod, Makefile, netlify.toml, Tiltfile
  • Prior art: 🌱 Bump Go to v1.22.2 #10452
• Bumps in Cluster API repo: ✨ Bump to controller-runtime v0.19 & controller-tools v0.16 #10803
  • controller-runtime & controller-tools
  • conversion-gen via CONVERSION_GEN_VER in Makefile
• Bump the Kubernetes version used in integration tests via KUBEBUILDER_ENVTEST_KUBERNETES_VERSION in Makefile
  • Note: This PR should be cherry-picked as well. It is part of this section as it depends on kubebuilder/controller-runtime releases and is not strictly necessary for Supporting managing and running on the new Kubernetes version.
  • Prior art: 🌱 Update kubebuilder envtest (1.24.2 -> 1.25.0) #7193
  • Prior art to release envtest binaries: ✨ Release envtest v1.31.0 controller-tools#1032
  • PR: 🌱 Bump envtest in Makefile to v1.31.0 #11085
  • release-1.8 PR: [release-1.8] 🌱 Bump envtest in Makefile to v1.31.0 #11086

Follow-ups after CR v0.19 bump:

• Simplify metric queries / Grafana dasbhoards (xref: ✨ Workqueue: Add controller label controller-runtime#2895)
  • Nothing to do, we only have simple queries in the controller-runtime dashboard like this workqueue_depth{pod=~"$Pod",name=~"$Controller"}. We should keep using name there for compatibility with controllers that are using CR versions <= v0.18.x
• Add e2e test coverage to verify via metrics that no panics occured (xref: ⚠ Recover panics per default controller-runtime#2905): ✨ Detect panics in e2e tests #11066
• Filter out finalizer warnings in controllers (xref: ⚠️ Remove options.WarningHandler controller-runtime#2903)
  • Opened an issue: Consider filtering out finalizer warnings in controller logs #11065

[sbueringer]

/assign @sbueringer
/assign @chrischdi

/triage accepted

Nothing to do until ~ July.

[kranurag7]

FYI, 1.31 beta is out now. Ref: https://github.com/kubernetes/kubernetes/releases/tag/v1.31.0-beta.0

Looking forward to help with few things here out of many. :)

[sbueringer]

Thx! Let's see how it goes. Not sure about the critical path to get v1.31 support out, because we have a very tight deadline, so I would like to do this closely coordinated with @chrischdi.

But there are a definitely other tasks on this issue that are not in the critical path (they'll be after Kubernetes 1.31 GA and CAPI v1.8.1 though)

[sivchari]

Bump quickstart and kind image references in CAPD

Hi. Can I take this task ?

[sbueringer]

This can only be done after the prerequisites are fulfilled:

Prerequisites:

• The target Kubernetes version is GA
• There is a new kind version with/or a new set of kind images for the target Kubernetes version

[sivchari]

@sbueringer
I understood, thanks. When the prerequisites are fulfilled, I'd contribute it. And I'm willing to help with other tasks. :)

[sbueringer]

Hey folks,
the following two tasks can be done now:

• Update our ProwJobs for the main branch to use the kubekins-e2e with the correct Kubernetes version
• Bump the Kubernetes version used in integration tests via KUBEBUILDER_ENVTEST_KUBERNETES_VERSION in Makefile

[sbueringer]

kind v0.24 was released (https://github.com/kubernetes-sigs/kind/releases/tag/v0.24.0)

So "Bump quickstart and kind image references in CAPD" is also unblocked

[sbueringer]

All done

Thx everyone!

/close

[k8s-ci-robot]

@sbueringer: Closing this issue.

In response to this:

All done

Thx everyone!

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.