PF_RING

Install PF_Ring

From Packages

For latest instructions see the Ntop install directions.

Add the Ntop Apt repository

apt-get install -y -q software-properties-common wget
add-apt-repository universe
wget https://packages.ntop.org/apt-stable/$(lsb_release -sr)/all/apt-ntop-stable.deb
apt install ./apt-ntop-stable.deb

Install the PF_Ring packages

apt-get update
apt-get install -y -q pfring

List the interfaces and check the driver model

pf_ringcfg --list-interfaces
Name: em1                  Driver: igb        [Supported by ZC]
Name: p1p2                 Driver: ixgbe      [Supported by ZC]
Name: p1p1                 Driver: ixgbe      [Supported by ZC]
Name: em2                  Driver: e1000e     [Supported by ZC]

Configure and load the driver specifying the driver model and (optionally) the number of RSS queues per interface:

pf_ringcfg --configure-driver e100e --rss-queues 1

Check that the driver has been successfully loaded by looking for ‘Running ZC’:

pf_ringcfg --list-interfaces
Name: em1                  Driver: igb        [Supported by ZC]
Name: p1p2                 Driver: ixgbe      [Supported by ZC]
Name: p1p1                 Driver: ixgbe      [Supported by ZC]
Name: em2                  Driver: e1000e     [Running ZC]

From Source

1. Install the PF_RING kernel Module

cd PF_RING/kernel
make
sudo make install

2. Insert the kernel module

See the docs for more options.

# sudo insmod ./pf_ring.ko [min_num_slots=N] [enable_tx_capture=1|0] [ enable_ip_defrag=1|0]
sudo insmod pf_ring.ko min_num_slots=65536

min_num_slots Minimum number of packets the kernel module should be able to enqueue (default – 4096).

3. Compile and run Zero Copy (ZC) drivers

# Determine the driver family
ethtool -i eth1 | grep driver
> e1000e

# Compile and load the corresponding driver
cd PF_RING/drivers/intel
make
cd e1000e/e1000e-*-zc/src
sudo ./load_driver.sh

4. Compile libpcap and libpfring

cd PF_RING/userland/lib
./configure && make
sudo make install
cd ../libpcap
./configure && make
sudo make install

5. Make the zero copy load balancer zbalance_ipc

cd PF_RING/userland/examples_zc/
make zbalance_ipc

Run Zero Copy Load Balancer

Using Systemd

A zbalance.service systemd service file can be found in sysconfig which relies on the configurations in sysconfig/conjure.conf to run the zbalance_ipc process.

1. Install the service file:

sudo cp <path_to_conjure>/conjure/sysconfig/zbalance.service /etc/systemd/system/
sudo systemctl enable zbalance

2. Configure the parameters

# numerical core on which to run zbalance 
ZBALANCE_CORE=1

# How to balance packets across queues
ZBALANCE_HASH_MODE=1

# The Zbalance_ipc cluster id that conjure will connect to 
CJ_CLUSTER_ID=97

# number of cores to launch detector threads on
CJ_CORECOUNT=6

# Interfaces on which to capture packets (pf_ring will take over the interface)
CJ_IFACE="enp2sf0,en2sf1"

3. start the systemd process

sudo systemctl start zbalance

Manually

Start the ZC load balancer establishing cluster and queues to interface with.

See the docs for more options

cd PF_R/userland/examples_zc
# sudo zbalance_ipc -i zc:eth1 -n $CORES -c $CLUSTER_NUM -g 1
sudo ./zbalance_ipc -i zc:eth1 -n 2 -c 10 -g 1 -m 1

# comma separate when load balancing from more than one interface
sudo ./zbalance_ipc -i zc:eth1,zc:eth0 -n 2 -c 10 -g 1 -m 1

-g is the core affinity for the capture/distribution thread

-c declares the ZC cluster ID

-n specifies the number of egress queues

-m selects the hash function (there are a few options available, or it is possible to write a custom one)

0: Round-Robin (default)
1: IP hash
...

NOTE: If listening on more than one interface with zbalance_ipc they MUST be comma separated. Using other methods can lead to transparent failure and packet loss.

See PF_RING Documentation for the latest information on how to run PF_RING ZC correctly.

Modifying or Removing PF_Ring

To completely remove the PF_Ring package and all kernel modules you can use a script provided by ntop. See the ntop help page for more details.

Modifying PF_ring Version

To update the version of PF_Ring in use the kernel module(s) associated with the old version will need to be removed before the new ones can be installed. This includes interface modules installed in step 3. above, kernel modules installed in step 1. and any programs running that depend on those kernel modules. The kernel modules must be removed in order of dependency as you cannot remove kernel modules that have other modules depending on and you cannot remove kernel modules in use by a process.

1. Stop zbalance_ipc or whatever ingest channel uses pf_ring

sudo systemctl stop zbalance

2. Stop any other pf_ring programs that might be using the kernel modules.

sudo systemctl stop nprobe

# a full list of programs that might be in use based on the ntop uninstall script
# cento e1000e-zc-dkms fm10k-zc-dkms i40e-zc-dkms ice-zc-dkms igb-zc-dkms ixgbevf-zc-dkms ixgbe-zc-dkms n2disk n2n nbox ndpi ndpi-dev nedge nprobe nprobe-agent nprobe-dev nprobes nscrub ntopng ntopng-data pfring pfring-dkms pfring-drivers-zc-dkms

3. Remove the interface drivers installed in step 3 (e1000e in this example but depends on your devices)

sudo rmmod e1000e

4. Ensure the pf_ring kernel module has no nothing else depending on it before removing it.

# if this is run as root and dependencies still exist this may give hints towards the driver or process dependencies. 
> sudo lsmod | grep pf_ring
pf_ring               729088  16 e1000e


# once there are no more dependencies you can remove the pf_ring kernel module.
> sudo lsmod | grep pf_ring
pf_ring               729088  0
> sudo rmmod pf_ring 

Now you are free to install new versions of the kernel module.