can not create pod on specific node any more

[hchyue]

I have a rke2 cluster with 3 master and 2 worker node.

node1 has 134 running pod
node2 has 273 running pod
calico info is :

no more pod can run on node2, events of new pod on node2 is:

I can not find the root cause, any help is appreciated.

[hchyue]

I find that no corresponding ipamhandle crd created

[hchyue]

calico-kube-controllers pod on node2. there is "healthz check failed" in the logs. In fact,etcd is running

[hchyue]

time in apiserver logs

[coutinhop]

node1 has 134 running pod
node2 has 273 running pod

@hchyue you seem to be pushing past the recommended limits of kubernetes itself: https://kubernetes.io/docs/setup/best-practices/cluster-large/

Furthermore, a /32 block size means you have only one IP address per block, which will surely have performance implications at this scale. Any specific reason for using that?

[hchyue]

We are conducting stress tests.

We use BGP mode. When the block size is not 32, pod with persistent IP addresses restart on other nodes,there is a blackhole route causing network unreachable to the pod from original node.

[caseydavenport]

persistent IP addresses restart on other nodes,there is a blackhole route causing network unreachable to the pod from original node.

This isn't expected behavior - Calico should advertise a /32 route that takes precedence over the blackhole route.