[BUG] ip address not be cleaned in vpc-nat-gw pod after eip is released

[hackerain]

Kube-OVN Version

master

Kubernetes Version

v1.29.3

Operation-system/Kernel Version

centos 9

Description

the ip address in vpc-nat-gw pod is not cleaned after eip is released

Steps To Reproduce

1. we have the following eips

[root@master-1 ~]# kubectl get eip
NAME                   IP            MAC                 NAT        NATGWDP                 READY
iptableseip-eesesquk   10.1.69.205   7a:f0:ad:cd:5f:64   fip        v-gw-383c4e1251c1194e   true
iptableseip-jntiodwf   10.1.69.206   b2:68:15:f3:9d:cd   fip        v-gw-383c4e1251c1194e   true
iptableseip-vzstoprc   10.1.69.202   ca:97:87:69:27:6a   fip        v-gw-383c4e1251c1194e   true
iptableseip-yqdmlwxn   10.1.69.203   d6:b1:b8:c5:d8:33   fip        v-gw-383c4e1251c1194e   true
kubem001eip            10.1.69.208   5a:50:38:b5:0f:6c   fip,snat   v-gw-383c4e1251c1194e   true

2. in the vpc-nat-gw pod

[root@master-1 ~]# kubectl exec -it -n kube-system vpc-nat-gw-v-gw-383c4e1251c1194e-0 -- bash
vpc-nat-gw-v-gw-383c4e1251c1194e-0:/kube-ovn# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: net1@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc htb state UP group default qlen 1000
    link/ether 2a:e5:31:9c:db:61 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.1.69.201/24 brd 10.1.69.255 scope global net1
       valid_lft forever preferred_lft forever
    inet 10.1.69.208/24 scope global secondary net1
       valid_lft forever preferred_lft forever
    inet 10.1.69.202/24 scope global secondary net1
       valid_lft forever preferred_lft forever
    inet 10.1.69.203/24 scope global secondary net1
       valid_lft forever preferred_lft forever
    inet 10.1.69.205/24 scope global secondary net1
       valid_lft forever preferred_lft forever
    inet 10.1.69.206/24 scope global secondary net1
       valid_lft forever preferred_lft forever
    inet6 fe80::28e5:31ff:fe9c:db61/64 scope link
       valid_lft forever preferred_lft forever
128: eth0@if129: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UP group default
    link/ether d2:1b:ee:11:d4:05 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.0.1.254/24 brd 10.0.1.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::d01b:eeff:fe11:d405/64 scope link
       valid_lft forever preferred_lft forever

3. now release one eip

[root@master-1 ~]# kubectl delete eip iptableseip-eesesquk
iptableseip.kubeovn.io "iptableseip-eesesquk" deleted

4. look up the eip list to check the eip(10.1.69.205) is released

[root@master-1 ~]# kubectl get eip
NAME                   IP            MAC                 NAT        NATGWDP                 READY
iptableseip-jntiodwf   10.1.69.206   b2:68:15:f3:9d:cd   fip        v-gw-383c4e1251c1194e   true
iptableseip-vzstoprc   10.1.69.202   ca:97:87:69:27:6a   fip        v-gw-383c4e1251c1194e   true
iptableseip-yqdmlwxn   10.1.69.203   d6:b1:b8:c5:d8:33   fip        v-gw-383c4e1251c1194e   true
kubem001eip            10.1.69.208   5a:50:38:b5:0f:6c   fip,snat   v-gw-383c4e1251c1194e   true

5. but the in the vpc-nat-gw pod the ip address is still on the nic net1

[root@master-1 ~]# kubectl exec -it -n kube-system vpc-nat-gw-v-gw-383c4e1251c1194e-0 -- bash
vpc-nat-gw-v-gw-383c4e1251c1194e-0:/kube-ovn# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: net1@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc htb state UP group default qlen 1000
    link/ether 2a:e5:31:9c:db:61 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.1.69.201/24 brd 10.1.69.255 scope global net1
       valid_lft forever preferred_lft forever
    inet 10.1.69.208/24 scope global secondary net1
       valid_lft forever preferred_lft forever
    inet 10.1.69.202/24 scope global secondary net1
       valid_lft forever preferred_lft forever
    inet 10.1.69.203/24 scope global secondary net1
       valid_lft forever preferred_lft forever
    inet 10.1.69.205/24 scope global secondary net1
       valid_lft forever preferred_lft forever
    inet 10.1.69.206/24 scope global secondary net1
       valid_lft forever preferred_lft forever
    inet6 fe80::28e5:31ff:fe9c:db61/64 scope link
       valid_lft forever preferred_lft forever
128: eth0@if129: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UP group default
    link/ether d2:1b:ee:11:d4:05 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.0.1.254/24 brd 10.0.1.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::d01b:eeff:fe11:d405/64 scope link
       valid_lft forever preferred_lft forever

6. look at the ovn-controller log(set log level to 3)

I1108 21:20:49.587167       6 vpc_nat_gw_eip.go:49] enqueue update iptables eip iptableseip-eesesquk
I1108 21:20:49.587315       6 vpc_nat_gw_eip.go:188] handle update iptables eip iptableseip-eesesquk
I1108 21:20:49.587721       6 net.go:436] expand exclude ips [10.1.69.1 10.1.69.2..10.1.69.200 10.1.69.224]
I1108 21:20:49.644331       6 vpc_nat_gw_eip.go:211] clean eip "iptableseip-eesesquk" in pod
I1108 21:20:49.644518       6 vpc_nat_gateway.go:651] bash /kube-ovn/nat-gateway.sh eip-del 10.1.69.0/24
I1108 21:20:49.818763       6 vpc_nat_gateway.go:665] ExecuteCommandInContainer stdOutput: eip-del 10.1.69.0/24
-A SNAT_FILTER -j SHARED_SNAT

we can see the eip del command is

bash /kube-ovn/nat-gateway.sh eip-del 10.1.69.0/24

this will not delete the ip address, the correct command should looks like:

bash /kube-ovn/nat-gateway.sh eip-del 10.1.69.205/24

Current Behavior

look at "Steps To Reproduce"

Expected Behavior

the ip address in vpc-nat-gw pod should be deleted after eip is released, or it will cause duplicated eip.

[dosubot]

Hey @hackerain! I'm here to help you with any bugs, questions, or contributions. Let me know how I can assist you.

It seems like you're experiencing an issue where the IP address in the vpc-nat-gw pod is not being cleaned up after an EIP is released. This is similar to a previously closed issue where IP resources were not being properly recycled ^[1]. The issue was resolved by a pull request that implemented a fix to delete IP CRDs based on the pod name.

In your case, the problem appears to be with the command used to delete the EIP. The command executed is:

bash /kube-ovn/nat-gateway.sh eip-del 10.1.69.0/24

However, the correct command should specify the exact IP address to be deleted, like this:

bash /kube-ovn/nat-gateway.sh eip-del 10.1.69.205/24

To resolve this, you may need to adjust the script or the logic that constructs the command to ensure it uses the specific IP address of the EIP being released. This will ensure that the IP address is correctly removed from the network interface in the vpc-nat-gw pod.

To continue talking to Dosu, mention @dosu.