skip to main content
10.1145/1028788.1028798acmconferencesArticle/Chapter ViewAbstractPublication PagesimcConference Proceedingsconference-collections
Article

Self-configuring network traffic generation

Published: 25 October 2004 Publication History

Abstract

The ability to generate repeatable, realistic network traffic is critical in both simulation and testbed environments. Traffic generation capabilities to date have been limited to either simple sequenced packet streams typically aimed at throughput testing, or to application-specific tools focused on, for example, recreating representative HTTP requests. In this paper we describe Harpoon, a new application-independent tool for generating representative packet traffic at the <i>IP flow level</i>. Harpoon generates TCP and UDP packet flows that have the same byte, packet, temporal and spatial characteristics as measured at routers in live environments. Harpoon is distinguished from other tools that generate statistically representative traffic in that it can <i>self-configure</i> by automatically extracting parameters from standard Netflow logs or packet traces. We provide details on Harpoon's architecture and implementation, and validate its capabilities in controlled laboratory experiments using configurations derived from flow and packet traces gathered in live environments. We then demonstrate Harpoon's capabilities in a router benchmarking experiment that compares Harpoon with commonly used throughput test methods. Our results show that the router subsystem load generated by Harpoon is significantly different, suggesting that this kind of test can provide important insights into how routers might behave under actual operating conditions.

References

[1]
Catalyst 6500 series switches. http://www.cisco.com/univer-cd/cc/td/doc/pro-duct/lan/cat-6000/index.htm. Accessed August 2004.]]
[2]
Cisco's IOS Netflow feature. http://www.cisco.com/-warp/-public/-732/-netflow. Accessed August 2004.]]
[3]
CoralReef: Passive network traffic monitoring and statistics collection. http://www.caida. org/tools/measurement/coralreef. Accessed August 2004.]]
[4]
Endace measurement systems. http://www.endace.com/. Accessed August 2004.]]
[5]
The eXpat XML parser. http://expat.sourceforge.net. Accessed August 2004.]]
[6]
The iperf TCP/UDP Bandwidth Measurement Tool. http://dast.nlanr.net/Projects/Iperf. Accessed August 2004.]]
[7]
Netflow services solutions guide (Netflow white paper). http://www.cisco.com/-univercd/cc/td/-doc/cisintwk/ intsolns/netflsol/-nfwhite.htm. Accessed August 2004.]]
[8]
Spirent Communications Inc. Adtech AX/4000 broadband test system. http://www.spirentcom.com/analy-sis/pro-duct_line.cfm?pl=1&WS=173&wt=2. Accessed August 2004.]]
[9]
SSFnet network simulator. http://www.ssfnet.org. Accessed August 2004.]]
[10]
The University of New Hampshire Interoperability Laboratory. http://www.iol.unh.edu. Accessed August 2004.]]
[11]
The Wisconsin Advanced Internet Laboratory. http://wail.cs.wisc.edu. Accessed August 2004.]]
[12]
UCB/LBNL/VINT Network Simulator - ns (version 2). http://www.isi.edu/nsnam/ns. Accessed August 2004.]]
[13]
Web polygraph. http://www.web-polygraph.org. Accessed August 2004.]]
[14]
XML-RPC home page. http://www.xmlrpc.org. Accessed August 2004.]]
[15]
Workshop on models, methods and tools for reproducible network research. http://www.acm.org/sigs/-sigcomm/sigcomm2003/-workshop/mometools, 2003.]]
[16]
P. Abry and D. Veitch. Wavelet analysis of long range dependent traffic. IEEE Transactions on Information Theory, 44(1):2--15, 1998.]]
[17]
C. Barakat, P. Thiran, G. Iannaccone, C. Diot, and P. Owezarski. Modeling Internet backbone traffic at the flow level. IEEE Transactions on Signal Processing (Special Issue on Networking), August 2003.]]
[18]
P. Barford and M. Crovella. Generating representative workloads for network and server performance evaluation. In Proceedings of ACM SIGMETRICS '98, pages 151--160, Madison, WI, June 1998.]]
[19]
P. Barford and M. Crovella. A performance evaluation of hyper text transfer protocols. In Proceedings of ACM SIGMETRICS '99, Atlanta, GA, May 1999.]]
[20]
S. Bradner. Benchmarking terminology for network interconnect devices. IETF RFC 1242, July 1991.]]
[21]
S. Bradner and J. McQuaid. Benchmarking methodology for network interconnect devices. IETF RFC 2544, March 1999.]]
[22]
T. Bu and D. Towsley. Fixed point approximation for TCP behavior in an AQM network. In Proceedings of ACM SIGMETRICS '01, San Diego, CA, June 2001.]]
[23]
Y. -C. Cheng, U. Hölzle, N. Cardwell, S. Savage, and G. M. Voelker. Monkey see, monkey do: A tool for TCP tracing and replaying. In Proceedings of the USENIX 2004 Conference, June 2004.]]
[24]
K. Claffy, G. Polyzos, and H. -W. Braun. Internet traffic flow profiling. Technical Report TR-CS93-328, University of California San Diego, November 1989.]]
[25]
W. Cleveland, D. Lin, and D. Sun. IP packet generation: Statistical models for TCP start times based on connection rate superposition. In Proceedings of ACM SIGMETRICS '00, Santa Clara, CA, June 2000.]]
[26]
M. Crovella and A. Bestavros. Self-similarity in World Wide Web traffic: Evidence and possible causes. IEEE/ACM Transactions on Networking, 5(6):835--846, December 1997.]]
[27]
N. Duffield, C. Lund, and M. Thorup. Estimating flow distributions from sampled flow statistics. In Proceedings of ACM SIGCOMM '03, Karlsruhe, Germany, August 2003.]]
[28]
A. Feldmann, A. Gilbert, P. Huang, and W. Willinger. Dynamics of IP traffic: A study of the role of variability and the impact of control. In Proceedings of ACM SIGCOMM '99, Boston, MA, August 1999.]]
[29]
A. Feldmann, A. Gilbert, and W. Willinger. Data networks as cascades: Investigating the multifractal nature of Internet WAN traffic. In Proceedings of ACM SIGCOMM '98, August 1998.]]
[30]
S. Floyd and E. Kohler. Internet research needs better models. In Hotnets-I, Princeton, NJ, October 2002.]]
[31]
S. Floyd and V. Paxson. Difficulties in simulating the Internet. IEEE/ACM Transactions on Networking, 9(4), August 2001.]]
[32]
M. Fomenkov, K. Keys, D. Moore, and K. Claffy. Longitudinal study of Internet traffic from 1998-2001: a view from 20 high performance sites. Technical report, Cooperative Association for Internet Data Analysis (CAIDA), 2002.]]
[33]
N. L. for Applied NetworkResearch. http://moat.nlanr.net/Datacube. Accessed August 2004.]]
[34]
C. Fraleigh, S. Moon, B. Lyles, C. Cotton, M. Khan, D. Moll, R. Rockell, T. Seely, and C. Diot. Packet-level traffic measurements from the Sprint IP backbone. IEEE Network, 2003.]]
[35]
S. Fredj, T. Bonald, A. Proutiere, G. Regnie, and J. Roberts. Statistical bandwidth sharing: A study of congestion at flow level. In Proceedings of ACM SIGCOMM '01, San Diego, CA, August 2001.]]
[36]
M. Fullmer and S. Romig. The OSU flow-tools package and Cisco NetFlow logs. In Proceedings of the USENIX Fourteenth System Administration Conference LISA XIV, New Orleans, LA, December 2000.]]
[37]
S. Jin and A. Bestavros. GISMO: Generator of Streaming Media Objects and Workloads. Performance Evaluation Review, 29(3), 2001.]]
[38]
W. Leland, M. Taqqu, W. Willinger, and D. Wilson. On the self-similar nature of Ethernet traffic (extended version). IEEE/ACM Transactions on Networking, pages 2:1--15, 1994.]]
[39]
R. Mandeville. Benchmarking terminology for LAN switching devices. IETF RFC 2285, February 1998.]]
[40]
R. Mandeville and J. Perser. Benchmarking methodology for LAN switching devices. IETF RFC 2889, August 2000.]]
[41]
D. Newman, G. Chagnot, and J. Perser. Internet core router test. http://www.light -reading.com/doc-ument.asp?site=test-ing&doc_id=4009, March 2001. Accessed August 2004.]]
[42]
K. Park and W. Willinger. Self-Similar Network Traffic and Performance Evaluation. Wiley Interscience, 2000.]]
[43]
V. Paxson. Measurements and Analysis of End-to-End Internet Dynamics. PhD thesis, University of California Berkeley, 1997.]]
[44]
V. Paxson and S. Floyd. Wide-area traffic: The failure of poisson modeling. IEEE/ACM Transactions on Networking, (3):226--244, June 1995.]]
[45]
D. Plonka. Flowscan: A network traffic flow reporting and visualization tool. In Proceedings of the USENIX Fourteenth System Administration Conference LISA XIV, New Orleans, LA, December 2000.]]
[46]
A. Turner. tcpreplay. http://tcpreplay.sourceforge.net/. Accessed August 2004.]]
[47]
S. Uhlig. Simulating interdomain traffic at the flow level. Technical Report Infonet-TR-2001-11, University of Namur, Institut d'Informatique, 2001.]]
[48]
A. Vahdat, K. Yocum, K. Walsh, P. Mahadevan, D. Kostic, J. Chase, and D. Becker. Scalability and accuracy in a large-scale network emulator. In Proceedings of 5th Symposium on Operating Systems Design and Implementation (OSDI), Boston, MA, December 2002.]]
[49]
B. White, J. Lepreau, L. Stoller, R. Ricci, S. Guruprasad, M. Newbold, M. Hibler, C. Barb, and A. Joglekar. An integrated experimental environment for distributed systems and networks. In Proceedings of 5th Symposium on Operating Systems Design and Implementation (OSDI), Boston, MA, December 2002.]]
[50]
W. Willinger, M. Taqqu, R. Sherman, and D. Wilson. Self-similarity through high-variability: Statistical analysis of Ethernet LAN traffic at the source level. IEEE/ACM Transactions on Networking, 5(1):71--86, February 1997.]]
[51]
M. Yajnik, S. Moon, J. Kurose, and D. Towsley. Measurement and modeling of temporal dependence in packet loss. In Proceedings of IEEE INFOCOM '99, New York, NY, March 1999.]]

Cited By

View all
  • (2024)Regional Features Conditioned Diffusion Models for 5G Network Traffic GenerationProceedings of the 32nd ACM International Conference on Advances in Geographic Information Systems10.1145/3678717.3691312(396-409)Online publication date: 29-Oct-2024
  • (2024)ForTT-Gen: Network Traffic Generator for Malware Forensics Analysis Training2024 12th International Symposium on Digital Forensics and Security (ISDFS)10.1109/ISDFS60797.2024.10527345(1-6)Online publication date: 29-Apr-2024
  • (2022)Introducing the CYSAS-S3 Dataset for Operationalizing a Mission-Oriented Cyber Situational AwarenessSensors10.3390/s2214510422:14(5104)Online publication date: 7-Jul-2022
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
IMC '04: Proceedings of the 4th ACM SIGCOMM conference on Internet measurement
October 2004
386 pages
ISBN:1581138210
DOI:10.1145/1028788
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 October 2004

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. network flows
  2. traffic generation

Qualifiers

  • Article

Conference

IMC04
Sponsor:
IMC04: Internet Measurement Conference
October 25 - 27, 2004
Taormina, Sicily, Italy

Acceptance Rates

Overall Acceptance Rate 277 of 1,083 submissions, 26%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)51
  • Downloads (Last 6 weeks)7
Reflects downloads up to 27 Dec 2024

Other Metrics

Citations

Cited By

View all

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media