Add support for parsing Eth-Over-GRE

[igorribeiroduarte]

I'm using two machines to reproduce this problem. One of them it's used just for sending the packets via tunnel and on the other one I'm running npcapextract and n2disk.

These are the steps to reproduce this issue:

Run n2disk on the first machine

n2disk1g -I -P /var/run/n2disk/n2disk.pid -G 1 -A index_folder -p 1024 -b 1024 -i ens18f0 --disk-limit 93% -t 15 -a -o /disco03 -o /disco04

Create a GRE tunnel on the second machine and replay the pcap to it

• sudo modprobe ip_gre
• sudo ip link add name ${tunnel_name} type gretap local 192.168.10.2 remote 192.168.10.3
• sudo ip link set ${tunnel_name} up
• sudo tcpreplay -i ${tunnel_name} --mbps 10 -K --loop 1 gre-within-gre.pcap

On the first machine, try to extract the desired packets with npcapextract using inner host and/or inner port

• npcapextract -u root -t index_folder -b '2020-05-25 16:18:56' -e '2020-05-25 16:19:31' -o test.pcap -f '(inner host 224.0.0.9 and inner port 520) and (inner host 3.3.3.2 and inner port 520)'
• npcapextract -u root -t index_folder -b '2020-05-25 16:18:56' -e '2020-05-25 16:19:31' -o test.pcap -f '(inner host 224.0.0.9)'
• npcapextract -u root -t index_folder -b '2020-05-25 16:18:56' -e '2020-05-25 16:19:31' -o test.pcap -f '(inner host 3.3.3.1)'
• npcapextract -u root -t index_folder -b '2020-05-25 16:18:56' -e '2020-05-25 16:19:31' -o test.pcap -f '(inner host 3.3.3.2)'

Neither of the above extractions worked

Also, when I run npcaprintindex on the machine where n2disk is running, i get:

1917) len: 200, vlan: 0, vlan_qinq: 0, ipv4, proto: 17, 192.168.10.2:5247 -> 192.168.10.3:5247, l7proto: Unknown/Unknown, not ip
1921) len: 200, vlan: 0, vlan_qinq: 0, ipv4, proto: 17, 192.168.10.2:49772 -> 192.168.10.3:9995, l7proto: Unknown/Unknown, not ip
1923) len: 200, vlan: 0, vlan_qinq: 0, ipv4, proto: 6, 192.168.10.2:38666 -> 192.168.10.3:9100, l7proto: Unknown/Unknown, not ip

As you can see, n2disk is not identifying the tunneled ipv4.

I can assure you that the problem is not with the tunnel, since I'm monitoring the network on the first machine and I can see the traffic I was replaying.

This is the pcap I used for these tests:
gre-within-gre.pcap.zip

n2disk version: n2disk v.3.4.200414 (r5191)
pfring version: 7.6.0 (7.6.0-stable:0e07b27c0d593174584de79e273470366a91d307)

[cardigliano]

@igorribeiroduarte I was unable to reproduce this so far, could you try with something similar to this below, using the same pcap?

n2disk1g -I -G 1 -A  /storage/n2disk/gre  -p 100 -b 200 --disk-limit 10% -t 15 -a -o /storage/n2disk/gre -i enp0s3

pfsend -i stack:enp0s3 -f pcap/gre-within-gre.pcap 

# npcapprintindex -i /storage/n2disk/gre/*/*.pcap.idx | tail
619) len: 162, vlan: 0, vlan_qinq: 0, ipv4, proto: 47, 86.106.164.150:0 -> 72.205.54.70:0, l7proto: Unknown/Unknown, tunneled ipv4, proto: 0, 10.10.13.2:0 -> 10.10.11.2:0
620) len: 162, vlan: 0, vlan_qinq: 0, ipv4, proto: 47, 72.205.54.70:0 -> 86.106.164.150:0, l7proto: Unknown/Unknown, tunneled ipv4, proto: 0, 10.10.11.2:0 -> 10.10.13.2:0
621) len: 114, vlan: 0, vlan_qinq: 0, ipv4, proto: 47, 86.106.164.150:0 -> 72.205.54.70:0, l7proto: Unknown/Unknown, tunneled ipv4, proto: 0, 10.10.13.2:0 -> 10.10.11.2:0

[igorribeiroduarte]

@cardigliano, using pfsend npcapprintindex showed tunneled ipv4:

145) len: 162, vlan: 0, vlan_qinq: 0, ipv4, proto: 47, 86.106.164.150:0 -> 72.205.54.70:0, l7proto: Unknown/Unknown, tunneled ipv4, proto: 0, 10.10.13.2:0 -> 10.10.11.2:0
146) len: 162, vlan: 0, vlan_qinq: 0, ipv4, proto: 47, 72.205.54.70:0 -> 86.106.164.150:0, l7proto: Unknown/Unknown, tunneled ipv4, proto: 0, 10.10.11.2:0 -> 10.10.13.2:0```

[cardigliano]

Could you provide one of the pcap files n2disk is capturing from live traffic? I guess your actual traffic does not have the same headers as gre-within-gre.pcap

[igorribeiroduarte]

Sure.

n2disk_pcap.pcap.zip

[cardigliano]

Please note this is not IP-over-GRE, this is Eth-over-GRE, which is not currently supported (we can add support for it)

[igorribeiroduarte]

Right, thank you!

[igorribeiroduarte]

I have another doubt.
I was testing npcapextract using pfsend on the same machine instead of tcpreplay on the other machine, as you asked. It seems to work fine and extract the correct packets if I use the ip's from the second GRE layer (10.10.11.2 and 10.10.13.2, for example) but I'm not able to extract packets using the most inner GRE layer (3.3.3.1 and 3.3.3.2). Is this the correct behavior?

Also, can I use the inner keyword to any extraction? On a way that I can use npcap extract ... -f 'inner host ${IP_ADDR}'be the packet encapsulated or not?

[igorribeiroduarte]

Is there any configuration on n2disk that allows me to index the most inner GRE layer instead of the first one?

[cardigliano]

n2disk indexes the first layer, we can add an option to index the deepest layer

[igorribeiroduarte]

Right, thanks.
And about my second question, can I use the inner keyword always?

[cardigliano]

inner should be used to look for the encapsulated layer