OpenFlow Error while running ip_loadbalancer along with l2_forwarding module

[xer0xe9]

hello,

I've been trying to run ip_loadbalancer module along with l2_learning on following mininet topology:

s1 = self.addSwitch('s1')
s2 = self.addSwitch('s2')
s3 = self.addSwitch('s3')

attacker = self.addHost('attacker', ip='10.0.0.1/24', mac='00:00:00:00:00:01')
victim = self.addHost('victim', ip='10.0.0.2/24', mac='00:00:00:00:00:02')
LBOne = self.addHost('LBOne' , ip='10.0.0.3/24', mac='00:00:00:00:00:03')
LBTwo = self.addHost('LBTwo' , ip='10.0.0.4/24', mac='00:00:00:00:00:04')

self.addLink(attacker, s1)
self.addLink(victim, s2)
self.addLink(s1, s3)
self.addLink(s2, s3)
self.addLink(LBOne, s2)
self.addLink(LBTwo, s2)

using this command from within the topology

Popen("/root/pox/pox.py log.level --DEBUG pox.forwarding.l2_learning pox.misc.ip_loadbalancer --ip=10.0.0.2 servers=10.0.0.3,10.0.0.4 --dpid=00-00-00-00-00-02", shell=True)

but getting this error:

DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:01.2 -> 00:00:00:00:00:02.1
INFO:iplb.00-00-00-00-00-02:Directing traffic to 10.0.0.3
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:02.1 -> 00:00:00:00:00:01.2
ERROR:openflow.of_01:[00-00-00-00-00-02 3] OpenFlow Error:
[00-00-00-00-00-02 3] Error: header:
[00-00-00-00-00-02 3] Error: version: 1
[00-00-00-00-00-02 3] Error: type: 1 (OFPT_ERROR)
[00-00-00-00-00-02 3] Error: length: 76
[00-00-00-00-00-02 3] Error: xid: 694
[00-00-00-00-00-02 3] Error: type: OFPET_BAD_REQUEST (1)
[00-00-00-00-00-02 3] Error: code: OFPBRC_BUFFER_EMPTY (7)
[00-00-00-00-00-02 3] Error: datalen: 64
[00-00-00-00-00-02 3] Error: 0000: 01 0e 00 68 00 00 02 b6 00 00 00 00 00 02 00 00 |...h............|
[00-00-00-00-00-02 3] Error: 0010: 00 00 00 01 00 00 00 00 00 02 ff ff 00 00 08 00 |................|
[00-00-00-00-00-02 3] Error: 0020: 00 06 00 00 0a 00 00 01 0a 00 00 02 8b d0 00 50 |...............P|
[00-00-00-00-00-02 3] Error: 0030: 00 00 00 00 00 00 00 00 00 00 00 0a 00 00 80 00 |................|
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:01.2 -> 00:00:00:00:00:02.1
INFO:iplb.00-00-00-00-00-02:Directing traffic to 10.0.0.4
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:01.2 -> 00:00:00:00:00:02.1
INFO:iplb.00-00-00-00-00-02:Directing traffic to 10.0.0.3
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:01.2 -> 00:00:00:00:00:02.1
INFO:iplb.00-00-00-00-00-02:Directing traffic to 10.0.0.4
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:01.2 -> 00:00:00:00:00:02.1
INFO:iplb.00-00-00-00-00-02:Directing traffic to 10.0.0.3
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:02.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:02.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:02.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:02.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:02.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:02.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:02.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:02.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:03.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:03.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:01.1 -> 00:00:00:00:00:02.2
ERROR:openflow.of_01:[00-00-00-00-00-02 3] OpenFlow Error:
[00-00-00-00-00-02 3] Error: header:
[00-00-00-00-00-02 3] Error: version: 1
[00-00-00-00-00-02 3] Error: type: 1 (OFPT_ERROR)
[00-00-00-00-00-02 3] Error: length: 76
[00-00-00-00-00-02 3] Error: xid: 729
[00-00-00-00-00-02 3] Error: type: OFPET_BAD_REQUEST (1)
[00-00-00-00-00-02 3] Error: code: OFPBRC_BUFFER_EMPTY (7)
[00-00-00-00-00-02 3] Error: datalen: 64
[00-00-00-00-00-02 3] Error: 0000: 01 0e 00 68 00 00 02 d9 00 00 00 00 00 03 00 00 |...h............|
[00-00-00-00-00-02 3] Error: 0010: 00 00 00 03 00 00 00 00 00 01 ff ff 00 00 08 00 |................|
[00-00-00-00-00-02 3] Error: 0020: 00 06 00 00 0a 00 00 03 0a 00 00 01 00 50 8b ce |.............P..|
[00-00-00-00-00-02 3] Error: 0030: 00 00 00 00 00 00 00 00 00 00 00 0a 00 00 80 00 |................|
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:04.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:01.1 -> 00:00:00:00:00:02.2
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:01.1 -> 00:00:00:00:00:02.2
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:03.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:03.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:04.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:04.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:03.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:03.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:04.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:04.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:03.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:03.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:04.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:04.2 -> 00:00:00:00:00:01.1
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:01.1 -> 00:00:00:00:00:02.2
DEBUG:forwarding.l2_learning:installing flow for 00:00:00:00:00:03.2 -> 00:00:00:00:00:01.1

I am not able to get around this issue. If I run ip_loadbalancer module alone, it works okay for a single switch topology but as soon as I add one or two more switches to the topology and try to run l2_learniing module (for forwarding traffic on s1 and s3) along with ip_loadbalancer (running on s2), this error starts popping up. Should I run different instances of pox controller and connect with switches respectively? OR is there any other workaround to get over this error.

Please provide any way to resolve this.

[MurphyMc]

Yeah, running two instances of POX would be a pretty straightforward way. One for the load balancer and one for the switches you want to be learning switches.

Off the top of my head, I think you could also probably get it working with a single controller by doing one of the following...

1. When you give a specific DPID to ip_loadbalancer, it checks whether a ConnectionUp is for the given IP and ignores it if it's not. You could modify l2_learning to perform the opposite logic in its own ConnectionUp handler -- so that it ignores it for the given DPID.

2. You could probably modify the ip_loadbalancer so that when it decides it wants to control the switch (in its ConnectionUp handler), it "eats" the ConnectionUp event by returning pox.lib.revent.EventHalt. Then you probably need to make sure to load the ip_loadbalancer and l2_learning components in the correct order (just swapping them around on the commandline... I think one order will work and the other one won't, but I don't know which off the top of my head... probably ip_loadbalancer should go first).

[MurphyMc]

The POX manual wouldn't show information on that error, since it's an OpenFlow error, not a POX error.

This error usually means that something like the following has occurred. The switch had buffered a packet and sent a message to the controller (e.g., a packet_in) which an ID for the buffer. In response, the controller sent some message which referred to the buffered packet by ID (e.g., a packet_out), but that by the time the switch got that message, the packet was no longer buffered. Typically this happens because the controller is not keeping up with the rate of packets -- it's a performance problem.

One place this shows up is when people introduce a loop into their topology without dealing with it, since this often results in an exponential number of packets. It also shows up when people make mistakes like putting a call to a blocking function into their code which means the controller is blocking instead of dealing with new packets quickly. But it also comes up sometimes just because there's too many packets for the controller to manage.

There are various ways to try to mitigate this. You can try making the controller faster (often, running it using PyPy is an easy way... or possibly try POX's --unthreaded-sh switch). If your switches are based on Open vSwitch, you might be able to use extensions so that not all connections are handled by the same controller. Or rewrite your controller in a faster language and with a faster controller framework than POX (NOX, for example, was very fast). But, generally speaking, if your controller is trying to handle a high rate of packet_ins, it's a problem for which OpenFlow may not be the best tool.