kubeadm: graduate WaitForAllControlPlaneComponents to Beta

[neolit123]

What type of PR is this?

/kind feature

What this PR does / why we need it:

• Set the feature gate to Beta and enabled by default.
• Make sure that the source of truth for which address/port to use for a component health check comes from the respective component static Pod manifest. That is done to comply with any user --patches that are applied on top
  of the ClusterConfiguration.

Which issue(s) this PR fixes:

xref

• graduate WaitForAllControlPlaneComponents to GA kubeadm#2907

Special notes for your reviewer:

first commit fixes circular dependency problems. check the commit message.

Does this PR introduce a user-facing change?

kubeadm: graduated the WaitForAllControlPlaneComponents feature gate to Beta. When checking the health status of a control plane component, make sure that the address and port defined as arguments in the respective component's static Pod manifest are used.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

[neolit123]

/triage accepted
/priority backlog

[neolit123]

/hold for review

[neolit123]

running some e2e:
https://github.com/neolit123/kubeadm-test/actions/runs/12772694792
https://github.com/neolit123/kubeadm-test/actions/runs/12772686860
https://github.com/neolit123/kubeadm-test/actions/runs/12788889731

[neolit123]

i believe we do have users that use the scheduler --config option and have no flags in the scheduler manifest.
like so:
https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/control-plane-flags/#scheduler-flags

for those users this new logic will use the default address/port.

as far as i can tell there is no way to set the --secrure-port and --bind-address as fields in the KubeSchedulerConfiguration?
https://kubernetes.io/docs/reference/config-api/kube-scheduler-config.v1
if that is somehow possible then the logic in this PR will not work and the user must also pass these flags to the scheduler.

[neolit123]

there are some really messy circular dependencies going on. will try to debug more tomorrow or later this week..
edit - or maybe it gets fixed by this new commit.

[neolit123]

there are some really messy circular dependencies going on. will try to debug more tomorrow or later this week.. edit - or maybe it gets fixed by this new commit.

reworked the logic to something much cleaner.

[neolit123]

running some e2e:
https://github.com/neolit123/kubeadm-test/actions/runs/12772694792
https://github.com/neolit123/kubeadm-test/actions/runs/12772686860
https://github.com/neolit123/kubeadm-test/actions/runs/12788889731

latest e2e test runs are green.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hashim21223445, neolit123

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• cmd/kubeadm/OWNERS [neolit123]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[pacoxu]

/cc @HirazawaUi

[HirazawaUi]

It seems that some duplicate code can be omitted.

func getControlPlaneComponents(podMap map[string]*v1.Pod, addressAPIServer string) ([]controlPlaneComponent, error) {
	var (
		// By default kubeadm deploys the kube-controller-manager and kube-scheduler
		// with --bind-address=127.0.0.1. This should match get{Scheduler|ControllerManager}Command().
		addressKCM       = "127.0.0.1"
		addressScheduler = "127.0.0.1"

		portAPIServer = fmt.Sprintf("%d", constants.KubeAPIServerPort)
		portKCM       = fmt.Sprintf("%d", constants.KubeControllerManagerPort)
		portScheduler = fmt.Sprintf("%d", constants.KubeSchedulerPort)

		errs []error
		result []controlPlaneComponent
	)

	type componentConfig struct {
		name        string
		podKey      string
		args        []string
		defaultAddr string
		defaultPort string
		endpoint    string
	}

	components := []componentConfig{
		{
			name:        "kube-apiserver",
			podKey:      constants.KubeAPIServer,
			args:        []string{argAdvertiseAddress, argPort},
			defaultAddr: addressAPIServer,
			defaultPort: portAPIServer,
			endpoint:    endpointLivez,
		},
		{
			name:        "kube-controller-manager",
			podKey:      constants.KubeControllerManager,
			args:        []string{argBindAddress, argPort},
			defaultAddr: addressKCM,
			defaultPort: portKCM,
			endpoint:    endpointHealthz,
		},
		{
			name:        "kube-scheduler",
			podKey:      constants.KubeScheduler,
			args:        []string{argBindAddress, argPort},
			defaultAddr: addressScheduler,
			defaultPort: portScheduler,
			endpoint:    endpointLivez,
		},
	}

	for _, component := range components {
		address, port := component.defaultAddr, component.defaultPort

		values, err := getControlPlaneComponentAddressAndPort(
			podMap[component.podKey],
			component.podKey,
			component.args,
		)
		if err != nil {
			errs = append(errs, err)
		}

		if len(values[0]) != 0 {
			address = values[0]
		}
		if len(values[1]) != 0 {
			port = values[1]
		}

		result = append(result, controlPlaneComponent{
			name: component.name,
			url:  fmt.Sprintf("https://%s/%s", net.JoinHostPort(address, port), component.endpoint),
		})
	}

	if len(errs) > 0 {
		return nil, utilerrors.NewAggregate(errs)
	}
	return result, nil
}

[neolit123]

i like this, i will update to use it like that.

[HirazawaUi]

Would it be better if we make kubeadm join provide the same error output as kubeadm init here? And make handleError a utility function.
ref:

kubernetes/cmd/kubeadm/app/cmd/phases/init/waitcontrolplane.go

Lines 102 to 113 in 0caa36c

	handleError := func(err error) error {
	context := struct {
	Error string
	Socket string
	}{
	Error: fmt.Sprintf("%v", err),
	Socket: data.Cfg().NodeRegistration.CRISocket,
	}
	kubeletFailTempl.Execute(data.OutputWriter(), context)
	return errors.New("could not initialize a Kubernetes cluster")
	}

[neolit123]

i think that's not a bad idea, but probably should be done in a separate PR after this one merges. you can log an issue in k/kubedam so that we don't forget. thanks