vxlanTunnelMACAddr not published in Node resource spec

[mgrove36]

Expected Behavior

A vxlanTunnelMACAddr key-value pair should be present in the spec section of the Node resource created by Calico when a calico/node instance is started.

Current Behavior

No vxlanTunnelMACAddr key-value pair is present. calicoctl get node master -o yaml gives:

apiVersion: projectcalico.org/v3
kind: Node
metadata:
  annotations:
    projectcalico.org/kube-labels: '{"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","kubernetes.io/arch":"amd64","kubernetes.io/hostname":"master","kubernetes.io/os":"linux","node-role.kubernetes.io/control-plane":"","node.kubernetes.io/exclude-from-external-load-balancers":""}'
  creationTimestamp: "2024-07-12T08:26:13Z"
  labels:
    beta.kubernetes.io/arch: amd64
    beta.kubernetes.io/os: linux
    kubernetes.io/arch: amd64
    kubernetes.io/hostname: master
    kubernetes.io/os: linux
    node-role.kubernetes.io/control-plane: ""
    node.kubernetes.io/exclude-from-external-load-balancers: ""
  name: master
  resourceVersion: "140164"
  uid: c777xxxx9fb9
spec:
  addresses:
  - address: 192.168.124.153/24
    type: CalicoNodeIP
  - address: 192.168.124.153
    type: InternalIP
  bgp:
    ipv4Address: 192.168.124.153/24
  ipv4VXLANTunnelAddr: 10.100.219.64
  orchRefs:
  - nodeName: master
    orchestrator: k8s
status: {}

Possible Solution

Steps to Reproduce (for bugs)

1. Deploy a new k8s cluster with kubeadm, using cri-dockerd and a Pod IP range of 10.100.0.0/16
2. Install Calico using the instructions from the documentation, in VXLANCrossSubnet mode.
3. Get a node resource with calicoctl get node node_name -o yaml

Context

The reference for the Node resource states that vxlanTunnelMACAddr should be populated by the system - and not added manually - so I would expect it to be present whenever Calico is running with VXLAN.

Your Environment

• Calico version:
  • Client Version: v3.28.0
  • Git commit: 413e6f5
  • Cluster Version: v3.28.0
  • Cluster Type: typha,kdd,k8s,operator,bgp,kubeadm
• Kubernetes version:
  • Client Version: v1.30.2
  • Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
  • Server Version: v1.30.2
• Operating System and version:
  • All nodes are Debian 12 with kernel 6.1.0 (Linux master 6.1.0-23-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.99-1 (2024-07-15) x86_64 GNU/Linux)

[cyclinder]

Hi @mgrove36, Thanks for the report.

Do you change the tunnel mode? Do you use the VXLANCrossSubnet mode when you install the calico?

[mgrove36]

Hey @cyclinder 👋
It was installed in VXLANCrossSubnet mode, and presented the issue. Switching to the VXLAN Always mode gave the same issue

[cyclinder]

Did you find anything suspicious in the calico-node logs? I will try to reproduce it.

[mgrove36]

Thanks. calico-node logs show it is correctly receiving the MACs via proto.VXLANTunnelEndpointUpdate update from calculation graph but that when initially loading the configuration (felix/daemon.go 391: Successfully loaded configuration.) VXLANTunnelMACAddr and VXLANTunnelMACAddrV6 are both empty strings.

[mgrove36]

And the API resources themselves never seem to publish the MACs at all

[caseydavenport]

Hm, that field should be set from the "projectcalico.org/VXLANTunnelMACAddr" annotation on the underlying Kubernetes Node object - do you see that it is present if you use kubectl to get the node?

[mgrove36]

Nope, it's not set

[mgrove36]

Has anyone been able to replicate this?

[caseydavenport]

I can confirm that I also do not see the MAC address annotation.

However, my VXLAN network seems to be functioning just fine. I believe this is because the MAC address is deterministically calculated when that annotation isn't present:

calico/felix/calc/vxlan_resolver.go

Lines 359 to 363 in 05f32ea

	// vtepMACForHost checks if there is new MAC present in host config.
	// If new MAC is present in host config, then vtepMACForHost returns the MAC present in host config else
	// vtepMACForHost calculates a deterministic MAC address based on the provided host.
	// The returned address matches the address assigned to the VXLAN device on that node.
	func (c *VXLANResolver) vtepMACForHost(nodename string, ipVersion int) string {

So, I think this is working as intended but the reference documentation isn't correct.

[mgrove36]

Makes sense, thanks for verifying. VXLAN works fine for me too - it was just that annotation missing. Is it worth updating the implementation so the annotation is published via the API server? This would be the same functionality that Flannel provides

[caseydavenport]

I'd rather just update the documentation to say that field is deprecated and kept around for legacy reasons (which I believe is the case). Unless there's a reason we want to have it written into the API?

[mgrove36]

Don't think so, was just a suggestion. Sounds good to me :)

[caseydavenport]

My colleagues just reminded me that this field is indeed used specifically to handle migration from flannel->Calico, in order to maintain the use of the existing tunnel MAC to avoid network disruptions.

[mgrove36]

Ah, makes sense!