High resolution packet capture with higher layer pass through on NetFPGA card

Features

• Timestamp UDP packets
• Accurate timestamp (nanosecond difference using the Dag card [15] as reference )
• No need for additional hardware
• No need for customized driver
• Wireshark compatible packets
• Timestamp is included right into the payload of the packets

HARDWARE

Timestamp module:

The timestamp module is based off the timestamp module used in another Monitoring System project[21] [24] . The timestamp module was added before the RX queues in order to timestamp the incoming packets as they arrive. Timestamps are generated from 64 bits counter driven by the 125 MHz NetFPGA clock, giving one increment every 8ns. Packets are timestamped as soon as the start frame delimiter is detected.

Pass through capability and Monitoring Module:

The objective of our packet-capture tool is to achieve the maximum accuracy in time-stamping packet arrivals, and at the same time, to allow packets continue their trip to the application layer. We call the latter feature as pass through capability. This feature allows the user to timestamp a packet arrival at both the data-link layer (actual arrival time) and at the application layer (software arrival time).

Fig 1: High-resolution timestamp packet-capture tool with pass-through capability.

The monitoring module located between the Output Port Lookup module and the Output Queues module as shown in the figure below, takes care of embedding the timestamp inside the packets. It detects the payload of packet (UDP) and replaces the first 64 bits of data with the Timestamp of the current packet.

Fig 2: Design Datapath

SOFTWARE (Download Source Package Here)

The software part of the system is made of two programs written in Perl programing language. The packets received by the NetFPGA must be recorded using the network analyzing software Wireshark [17] . Using the first perl program (netfpga.pl) to analyze each packet saved by wireshark , a text file containing the identification number and the timestamp (at hardware (NetFPGA) level) is generated for all packets. The second perl program requires two files: one, containing packets stored using Dag Card and the second one containing packets stored using NetFPGA. The purpose of this program is to subtract the timestamp recorded by dagcard and NetFPGA , given that both system sniff the same set of packets simultaneously for a certain amount of time.

Fig 3: Setup for capturing packets on NetFPGA and dag card simultaneously

The program only does the subtraction if the packets match (IP identification number must be the same). If the result is positive that means that the Dag card took more time to record the packet, if negative, then the NetFPGA took more time to record the packet. If it is zero, it means that both took the same amount of time. How to use it?

** NetFPGA source package (Download Here) **

1. Copy the folder (project package )nic_monitor into to the directory “~/NetFPGA/project/”
2. Copy the bitfile (nic_monitor.bit) into the directory “~/NetFPGA/bitfiles/” Download the bitfile to the NetFPGA board
3. Open the terminal and use the command “cpci_reprogram.pl --all”
4. Type “nf_download ~/NetFPGA/bitfiles/nic_monitor.bit” to download the design on the board
5. Make sure the NetFPGA is setup like in fig 2
6. Configure the NetFPGA port to an ip address “ifconfig nf2c0 ip address”
7. You can start capturing using wireshark

Record timestamp from captured packets

1. Save captured packets from Wireshark

2. While wireshark is still displaying the captured packet

   •	Click on file
   •	Then click export packets dissection
   •	 as XML – “PDML”
   •	Save the packets as NetFPGA.xml inside directory “~/NetFPGA/projects/nic_monitor/sw/perl/”
   •	Go to terminal and change directory to “~/NetFPGA/projects/nic_monitor/sw/perl/”
   •	Type perl netfpga.pl >> “file name”  to save the output of the program in a file
   •	The output file will contain the timestamp and ip identification number of the packets
   

3. If capture was done with dag card as well repeat the first 3 steps from 2 and save packets as dag.xml then run the dag.pl file and follow the rest of the instruction from 2

4. If you want to see the difference of time between dag card and NetFPGA, make sure u have both NetFPGA.xml and dag.xml under sw/perl/ directory and run subtractor.pl .

Conclusion

After all the experiments and analysis we have conducted, we can attribute the existence of delay to the sniffer and the difference of clock rates between the two devices. In addition to this, another factor to consider for the comparison between these two devices is the difference in their architecture. Ultimately we can confidently rely on our NetFPGA design to perform good network measurement since it comparisons with the dag card are just some few nanosecond apart.

REFERENCES

The work is described in: Yaovi E. Kwasi, Roberto Rojas-Cessa "High-resolution hardware-based packet capture with higher-layer pass-through on NetFPGA card"

[1] V. Jacobson, C. Leres, and S. McCanne, “The tcpdump manual page,” Lawrence Berkeley Laboratory, Berkeley, CA, 1989.

[2] G. Combs et al., “Wireshark,” Web page: http://www. wireshark. org/last modified, pp. 12–02, 2007.

[3] V. Jacobson, C. Leres, and S. McCanne, “libpcap, lawrence berkeley laboratory, berkeley, ca,” Initial public release June, 1994.

[4] F. Schneider and J. Wallerich, “Performance evaluation of packet capturing systems for high-speed networks,” in Proceedings of the 2005 ACM conference on Emerging network experiment and technology. ACM, 2005, pp. 284–285.

[5] R. W. Herrell and T. P. Morrissey, “User scheduled direct memory access using virtual addresses,” Apr. 5 1994, uS Patent 5,301,287.

[6] J. C. Mogul and K. Ramakrishnan, “Eliminating receive livelock in an interrupt-driven kernel,” ACM Transactions on Computer Systems, vol. 15, no. 3, pp. 217–252, 1997.

[7] L. Rizzo, “Device polling support for freebsd,” in BSDConEurope Conference, 2001.

[8] L. Deri, “ncap: Wire-speed packet capture and transmission,” in End-to- End Monitoring Techniques and Services, 2005. Workshop on. IEEE, 2005, pp. 47–55.

[9] V. Corey, C. Peterman, S. Shearin, M. S. Greenberg, and J. Van Bokkelen,

[10] K. Salehin and R. Rojas-Cessa, “Active scheme to measure throughput of wireless access link in hybrid wired-wireless network,” Wireless Communications Letters, IEEE, vol. 1, no. 6, pp. 645–648, 2012.

[11] K. M. Salehin and R. Rojas-Cessa, “Ternary-search-based scheme to measure link available-bandwidth in wired networks,” in Global Telecommunications Conference (GLOBECOM 2010), 2010 IEEE. IEEE, 2010, pp. 1–5.

[12] “Measurement of packet processing time of an internet host using asynchronous packet capture at the data-link layer,” in IEEE International Conference on Communications, 2013, p. 5pp.

[13] G. Iannaccone, C. Diot, I. Graham, and N. McKeown, “Monitoring very high speed links,” in Proceedings of the 1st ACM SIGCOMM Workshop on Internet Measurement. ACM, 2001, pp. 267–271.

[14] J. Micheel, S. Donnelly, and I. Graham, “Precision timestamping of network packets,” in Proceedings of the 1st ACM SIGCOMM Workshop on Internet Measurement. ACM, 2001, pp. 273–277.

[15] D. Endace, “Network monitoring interface,” URL¡ http://www. endace. com.

[16] T. Wolf, R. Ramaswamy, S. Bunga, and N. Yang, “An architecture for distributed real-time passive network measurement,” in Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, 2006. MASCOTS 2006. 14th IEEE International Symposium on. IEEE, 2006, pp. 335–344.

[17] G. Combs et al., “Wireshark,” Web page: http://www. wireshark. org/last modified, pp. 12–02, 2007.

[18] NetFPGA, http://www.NetFPGA.org.

[19] G. Antichi, D. J. Miller, and S. Giordano, “An open-source hardware module for high-speed network monitoring on netfpga,” in European NetFPGA Developers Workshop, 2010.

[20] G. Watson, N. McKeown, and M. Casado, “Netfpga: A tool for network research and education,” in 2nd workshop on Architectural Research using FPGA Platforms (WARFP), vol. 3, 2006.

[21] G. Antichi, S. Giordano, D. J. Miller, and A. W. Moore, “Enabling open-source high speed network monitoring on netfpga,” in Network Operations and Management Symposium (NOMS), 2012 IEEE. IEEE, 2012, pp. 1029–1035.

[22] S. F. Donnelly, “High precision timing in passive measurements of data networks,” Ph.D. dissertation, Citeseer, 2002.

[23] P. Saul, “Direct digital synthesis,” Circuits and systems tutorials, p. 393, 1996.

[24] I. X. paper pending-please check, http://github.com/Caustic/NetFPGAwiki/ wiki/MonitoringSystem.

[25] M. System, http://github.com/Caustic/NetFPGAwiki/ wiki/MonitoringSystem.

[26] V. Jacobson, C. Leres, and S. McCanne, “pcap-packet capture library,” UNIX man page, 2001.

[27] A. Tirumala, F. Qin, J. Dugan, J. Ferguson, and K. Gibbs, “Iperf: The tcp/udp bandwidth measurement tool,” htt p://dast. nlanr. net/Projects, 2005.

[28] N. S. Black Box, “10/100/1000 Black Box copper tap,” http://www. blackbox. com.

[29] http://en.wikipedia.org/wiki/iperf