RKE2ControlPlane.spec.agentConfig.nodeAnnotations does not work with CAPO

[baburciu]

What happened:

While trying to confirm RKE2 node annotations feature (introduced in #177, for the use-case described by #155) we hit a strange behavior where the CP nodes in OpenStack infra lack the annotations:

[git:longhorn-node-annotation]root@bootstrap-vm:sylva-core # kubectl get rke2controlplane management-cluster-control-plane -o yaml | yq .spec.agentConfig.nodeAnnotations
node.longhorn.io/default-disks-config: '[ { "path":"/var/lib/longhorn", "allowScheduling":true}, { "name":"fast-ssd-disk", "path":"/mnt/extra", "allowScheduling":false, "storageReserved":10485760, "tags":[ "ssd", "fast" ] }]'
node.longhorn.io/default-node-tags: '["fast","storage"]'
sylva.org/test-annotation-scope: ci
[git:longhorn-node-annotation]root@bootstrap-vm:sylva-core # kubectl --kubeconfig management-cluster-kubeconfig get nodes management-cluster-cp-e4e221327c-j5kw5 -o yaml | yq .metadata.annotations | grep -E "longhorn|test"
rke2.io/node-args: '["server","--cluster-cidr","10.42.0.0/16","--cni","calico","--kubelet-arg","anonymous-auth=false","--kubelet-arg","provider-id=openstack:///8bf6025d-19e1-4de0-8c25-ea7331ba204d","--node-label","node.longhorn.io/create-default-disk=config","--node-label","sylva.org/annotate-node-from-label=true","--profile","cis-1.23","--service-cidr","10.43.0.0/16","--tls-san","192.168.128.112","--token","********"]'
[git:longhorn-node-annotation]root@bootstrap-vm:sylva-core #

What did you expect to happen:

Have the annotations present on the CP node.

How to reproduce it:

• Use an

apiVersion: controlplane.cluster.x-k8s.io/v1alpha1
kind: RKE2ControlPlane
metadata:
  name: workload-cluster-control-plane
  namespace: test
spec:
  replicas: 1
  infrastructureRef:
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha6
    kind: OpenStackMachineTemplate
    name: workload-cluster-cp-52479526a7
  agentConfig:
    kubelet:
      extraArgs:
      - provider-id=openstack:///{{ ds.meta_data.uuid }}
    nodeAnnotations:
      node.longhorn.io/create-default-disk: config
      node.longhorn.io/default-disks-config: vdb
      node.longhorn.io/default-node-tags: '["fast","storage"]'

• and then check the created node (OpenStack machine) annotations:

 NODE_NAME=$(kubectl get osm -o jsonpath='{ $.items[?(@.metadata.annotations.cluster\.x-k8s\.io\/cloned-from-name == "workload-cluster-cp-52479526a7")].metadata.name}')
 kubectl --kubeconfig created-cluster-kubeconfig get nodes $NODE_NAME -o yaml | yq .metadata.annotations

Anything else you would like to add:
Same code works for Metal3 infra.

Environment:

• rke provider version: v0.2.2
• OS (e.g. from /etc/os-release): Ubuntu 22.04.3 LTS (bootstrapped node)

[matrohon]

@richardcase do you any feedback about that?

[richardcase]

Not yet i'm afraid @matrohon . I will add this to our project board so it has greatly visibility.

[Danil-Grigorev]

May potentially be fixed by #258