research-article

Reproducible network experiments using container-based emulation

Authors:

Nikhil Handigol,

Brandon Heller,

Vimalkumar Jeyakumar,

Nick McKeownAuthors Info & Claims

CoNEXT '12: Proceedings of the 8th international conference on Emerging networking experiments and technologies

Pages 253 - 264

https://doi.org/10.1145/2413176.2413206

Published: 10 December 2012 Publication History

Abstract

In an ideal world, all research papers would be runnable: simply click to replicate all results, using the same setup as the authors. One approach to enable runnable network systems papers is Container-Based Emulation (CBE), where an environment of virtual hosts, switches, and links runs on a modern multicore server, using real application and kernel code with software-emulated network elements. CBE combines many of the best features of software simulators and hardware testbeds, but its performance fidelity is unproven.

In this paper, we put CBE to the test, using our prototype, Mininet-HiFi, to reproduce key results from published network experiments such as DCTCP, Hedera, and router buffer sizing. We report lessons learned from a graduate networking class at Stanford, where 37 students used our platform to replicate 18 published results of their own choosing. Our experiences suggest that CBE makes research results easier to reproduce and build upon.

References

[1]

D. E. Knuth. Literate Programming. The Computer Journal {Online}, 27:97--111, 1984.

Digital Library

[2]

J. Claerbout. Electronic documents give reproducible research a new meaning. Proc. 62nd Ann. Int. Meeting of the Soc. of Exploration Geophysics, pages 601--604, 1992.

[3]

J. B. Buckheit and D. L. Donoho. Wavelab and reproducible research. Time, 474:55--81, 1995.

[4]

P. Vandewalle, J. Kovacevic, and M. Vetterli. Reproducible research. Computing in Science Engineering, pages 5--7, 2008.

[5]

Amazon Elastic Compute Cloud. http://aws.amazon.com.

[6]

The network simulator - ns-2. http://nsnam.isi.edu/nsnam/.

[7]

The ns-3 network simulator. http://www.nsnam.org/.

[8]

OPNET modeler. http://www.opnet.com/solutions/network_rd/modeler.html.

[9]

Global Environment for Network Innovations. http://www.geni.net/.

[10]

B. Chun, D. Culler, T. Roscoe, A. Bavier, L. Peterson, M. Wawrzoniak, and M. Bowman. Planetlab: an overlay testbed for broad-coverage services. SIGCOMM Computer Communication Review, 33(3):3--12, 2003.

Digital Library

[11]

J. DeHart, F. Kuhns, J. Parwatikar, J. Turner, C. Wiseman, and K. Wong. The open network laboratory. In SIGCSE '06 Technical Symposium on Computer Science Education, pages 107--111. ACM, 2006.

Digital Library

[12]

Emulab - network emulation testbed. http://emulab.net/.

[13]

M. Al-Fares, S. Radhakrishnan, B. Raghavan, N. Huang, and A. Vahdat. Hedera: Dynamic flow scheduling for data center networks. In NSDI '10. USENIX, 2010.

Digital Library

[14]

M. Alizadeh, A. Greenberg, D. A. Maltz, J. Padhye, P. Patel, B. Prabhakar, S. Sengupta, and M. Sridharan. Data Center TCP (DCTCP). In SIGCOMM '10, pages 63--74. ACM, 2010.

Digital Library

[15]

N. Beheshti, Y. Ganjali, R. Rajaduray, D. Blumenthal, and N. McKeown. Buffer sizing in all-optical packet switches. In Optical Fiber Communication Conference. Optical Society of America, 2006.

[16]

D. Gupta, K. V. Vishwanath, and A. Vahdat. DieCast: Testing distributed systems with an accurate scale model. In NSDI '08, pages 407--421. USENIX, 2008.

Digital Library

[17]

Open vSwitch: An open virtual switch. http://openvswitch.org/.

[18]

M. Hibler, R. Ricci, L. Stoller, J. Duerig, S. Guruprasad, T. Stack, K. Webb, and J. Lepreau. Large-scale virtualization in the Emulab network testbed. In USENIX '08 Annual Technical Conference, pages 113--128, Berkeley, CA, USA, 2008. USENIX.

Digital Library

[19]

M. Pizzonia and M. Rimondini. Netkit: easy emulation of complex networks on inexpensive hardware. In International Conference on Testbeds and research infrastructures for the development of networks & communities, TridentCom '08, pages 7:1--7:10, Brussels, Belgium, 2008. ICST.

Digital Library

[20]

S. Bhatia, M. Motiwala, W. Muhlbauer, Y. Mundada, V. Valancius, A. Bavier, N. Feamster, L. Peterson, and J. Rexford. Trellis: a platform for building flexible, fast virtual networks on commodity hardware. In CoNEXT '08, pages 72:1--72:6. ACM, 2008.

Digital Library

[21]

J. Ahrenholz, C. Danilov, T. Henderson, and J. Kim. CORE: A real-time network emulator. In Military Communications Conference, MILCOM '08, pages 1 --7. IEEE, 2008.

[22]

B. Lantz, B. Heller, and N. McKeown. A network in a laptop: rapid prototyping for software-defined networks. In HotNets '10, pages 19:1--19:6. ACM, 2010.

Digital Library

[23]

S. Soltesz, H. Pötzl, M. Fiuczynski, A. Bavier, and L. Peterson. Container-based operating system virtualization: a scalable, high-performance alternative to hypervisors. ACM SIGOPS Operating Systems Review, 41(3):275--287, 2007.

Digital Library

[24]

M. Zec and M. Mikuc. Operating system support for integrated network emulation in IMUNES. In Workshop on Operating System and Architectural Support for the on demand IT InfraStructure (OASIS), 2004.

[25]

cgroups. http://www.kernel.org/doc/Documentation/cgroups/cgroups.txt.

[26]

P. Turner, B. B. Rao, and N. Rao. CPU bandwidth control for CFS. In Linux Symposium '10, pages 245--254, 2010.

[27]

Handigol, N., Heller, B., Jeyakumar, V., Lantz, B., and McKeown, N. CSTR 2012-02 Mininet performance fidelity benchmarks. http://hci.stanford.edu/cstr/reports/2012-02.pdf.

[28]

Linux Trace Toolkit - next generation. http://lttng.org/.

[29]

D. Gupta, K. Yocum, M. McNett, A. Snoeren, A. Vahdat, and G. Voelker. To infinity and beyond: time warped network emulation. In SOSP '05, pages 1--2. ACM, 2005.

Digital Library

[30]

lxc linux containers. http://lxc.sf.net.

[31]

K. Ramakrishnan and S. Floyd. A proposal to add explicit congestion notification (ECN) to IP. Technical report, RFC 2481, January 1999.

Digital Library

[32]

DCTCP patches. http://www.stanford.edu/"alizade/Site/DCTCP.html.

[33]

D. Thaler and C. Hopps. Multipath issues in unicast and multicast next-hop selection. Technical report, RFC 2991, November 2000.

Digital Library

[34]

Ripcord-Lite for POX: A simple network controller for OpenFlow-based data centers. https://github.com/brandonheller/riplpox.

[35]

G. Appenzeller, I. Keslassy, and N. McKeown. Sizing router buffers. In SIGCOMM '04, pages 281--292. ACM, 2004.

Digital Library

[36]

J. Lockwood, N. McKeown, G. Watson, G. Gibb, P. Hartke, J. Naous, R. Raghuraman, and J. Luo. NetFPGA -- an open platform for gigabit-rate network switching and routing. In Microelectronic Systems Education, MSE '07, pages 160--161. IEEE, 2007.

Digital Library

[37]

K. Nichols and V. Jacobson. Controlling queue delay. Communications of the ACM, 55(7):42--50, 2012.

Digital Library

[38]

M. Alizadeh, A. Kabbani, T. Edsall, B. Prabhakar, A. Vahdat, and M. Yasuda. Less is more: Trading a little bandwidth for ultra-low latency in the data center. In NSDI '12. USENIX, 2012.

Digital Library

[39]

C. Raiciu, C. Paasch, S. Barre, A. Ford, M. Honda, F. Duchene, O. Bonaventure, and M. Handley. How hard can it be? designing and implementing a deployable multipath TCP. In NSDI '12, pages 29--29. USENIX, 2012.

Digital Library

[40]

P. Prakash, A. Dixit, Y. Hu, and R. Kompella. The TCP outcast problem: Exposing unfairness in data center networks. In NSDI '12. USENIX, 2012.

Digital Library

[41]

A. Singla, C.-Y. Hong, L. Popa, and P. B. Godfrey. Jellyfish: Networking data centers randomly. In NSDI '12. USENIX, 2012.

Digital Library

[42]

N. Dukkipati, T. Refice, Y. Cheng, J. Chu, T. Herbert, A. Agarwal, A. Jain, and N. Sutin. An argument for increasing TCP's initial congestion window. SIGCOMM Computer Communication Review, 40(3):27--33, 2010.

Digital Library

[43]

V. Vasudevan, A. Phanishayee, H. Shah, E. Krevat, D. Andersen, G. Ganger, G. Gibson, and B. Mueller. Safe and effective fine-grained TCP retransmissions for datacenter communication. SIGCOMM Computer Communication Review, 39(4):303--314, 2009.

Digital Library

[44]

C. Guo, H. Wu, K. Tan, L. Shi, Y. Zhang, and S. Lu. DCell: a scalable and fault-tolerant network structure for data centers. In SIGCOMM '08. ACM, 2008.

Digital Library

[45]

N. Dukkipati and N. McKeown. Why flow-completion time is the right metric for congestion control. SIGCOMM Computer Communication Review, 36(1):59--62, 2006.

Digital Library

[46]

S. Savage, N. Cardwell, D. Wetherall, and T. Anderson. TCP congestion control with a misbehaving receiver. SIGCOMM Computer Communication Review, 29(5):71--78, 1999.

Digital Library

[47]

S. Floyd and V. Jacobson. Random Early Detection gateways for congestion avoidance. IEEE/ACM Transactions on Networking, 1(4):397--413, 1993.

Digital Library

[48]

C. Raiciu, S. Barre, C. Pluntke, A. Greenhalgh, D. Wischik, and M. Handley. Improving datacenter performance and robustness with multipath TCP. In SIGCOMM Computer Communication Review. ACM, 2011.

Digital Library

[49]

The OpenFlow switch. http://www.openflow.org.

[50]

M. Carson and D. Santay. NIST Net -- a Linux-based network emulation tool. SIGCOMM Computer Communication Review, 33(3):111--126, 2003.

Digital Library

[51]

M. Carbone and L. Rizzo. Dummynet revisited. SIGCOMM Computer Communication Review, 40(2):12--20, 2010.

Digital Library

[52]

Linux network emulation module. http://www.linuxfoundation.org/collaborate/workgroups/networking/netem.

[53]

P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield. Xen and the art of virtualization. In SOSP '03, pages 164--177. ACM, 2003.

Digital Library

[54]

A. Vahdat, K. Yocum, K. Walsh, P. Mahadevan, D. Kostić, J. Chase, and D. Becker. Scalability and accuracy in a large-scale network emulator. In OSDI '02, pages 271--284. USENIX, 2002.

Digital Library

[55]

OpenFlow Virtual Machine Simulation. http://www.openflow.org/wk/index.php/OpenFlowVMS.

[56]

E. Weingärtner, F. Schmidt, H. Vom Lehn, T. Heer, and K. Wehrle. Slicetime: A platform for scalable and accurate network emulation. In NSDI '11, volume 3. ACM, 2011.

Digital Library

[57]

Z. Tan, K. Asanovic, and D. Patterson. An FPGA-based simulator for datacenter networks. In Exascale Evaluation and Research Techniques Workshop (EXERT '10), at ASPLOS '10. ACM, 2010.

[58]

S. Tripathi, N. Droux, K. Belgaied, and S. Khare. Crossbow virtual wire: network in a box. In LISA '09, pages 4--4. USENIX, 2009.

Digital Library

Cited By

Gaffurini MFlammini AFerrari PFernandes Carvalho DGodoy ESisinni E(2024)End-to-End Emulation of LoRaWAN Architecture and Infrastructure in Complex Smart City Scenarios Exploiting ContainersSensors10.3390/s2407202424:7(2024)Online publication date: 22-Mar-2024
https://doi.org/10.3390/s24072024
De Coninck Q(2024)Core QUIC: Enabling Dynamic, Implementation-Agnostic Protocol Extensions2024 IFIP Networking Conference (IFIP Networking)10.23919/IFIPNetworking62109.2024.10619827(640-646)Online publication date: 3-Jun-2024
https://doi.org/10.23919/IFIPNetworking62109.2024.10619827
Han SWang SZhong CZhou DHuang TWei LTan C(2024)Openlab: A One-Stop Service Platform for Software-Defined Networks EmulationProceedings of the 2024 SIGCOMM Workshop on Formal Methods Aided Network Operation10.1145/3672199.3673893(45-47)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3672199.3673893
Show More Cited By

Index Terms

Reproducible network experiments using container-based emulation
1. Networks
  1. Network architectures

Recommendations

Virtual Time Machine for Reproducible Network Emulation
SIGSIM-PADS '19: Proceedings of the 2019 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation

Reproducing network emulation experiments on diverse physical platforms with varying computation and communication resources is non-trivial. Many state-of-the-art network emulation testbeds do not guarantee timing fidelity. Consequently, results ...
An Extended Virtual Network Functions Manager Architecture to Support Container
ICISS '18: Proceedings of the 1st International Conference on Information Science and Systems

Network function virtualization (NFV) is a network architecture concept that decouples network functions from hardware. With NFV, virtual network functions (VNFs) can easily be instantiated and deployed on standard servers using virtual machines (VMs). ...
A hardware-in-the-loop emulation testbed for high fidelity and reproducible network experiments
WSC '17: Proceedings of the 2017 Winter Simulation Conference

The transformation of innovative research ideas to production systems is highly dependent on the capability of performing realistic and reproducible network experiments. In this work, we present a network testbed consisting of container-based network ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CoNEXT '12: Proceedings of the 8th international conference on Emerging networking experiments and technologies

December 2012

384 pages

ISBN:9781450317757

DOI:10.1145/2413176

General Chairs:
Chadi Barakat
INRIA, Sophia Antipolis, France
,
Renata Teixeira
UPMC Sorbonne Universités and CNRS, Paris, France
,
Program Chairs:
K. K. Ramakrishnan
AT&T Labs Research, Florham Park, NJ, USA
,
Patrick Thiran
EPFL, Lausanne, Swizerland

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCOMM: ACM Special Interest Group on Data Communication

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 December 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CoNEXT '12

Sponsor:

SIGCOMM

CoNEXT '12: Conference on emerging Networking Experiments and Technologies

December 10 - 13, 2012

Nice, France

Acceptance Rates

Overall Acceptance Rate 198 of 789 submissions, 25%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

375
Total Citations
View Citations
1,927
Total Downloads

Downloads (Last 12 months)105
Downloads (Last 6 weeks)12

Reflects downloads up to 06 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gaffurini MFlammini AFerrari PFernandes Carvalho DGodoy ESisinni E(2024)End-to-End Emulation of LoRaWAN Architecture and Infrastructure in Complex Smart City Scenarios Exploiting ContainersSensors10.3390/s2407202424:7(2024)Online publication date: 22-Mar-2024
https://doi.org/10.3390/s24072024
De Coninck Q(2024)Core QUIC: Enabling Dynamic, Implementation-Agnostic Protocol Extensions2024 IFIP Networking Conference (IFIP Networking)10.23919/IFIPNetworking62109.2024.10619827(640-646)Online publication date: 3-Jun-2024
https://doi.org/10.23919/IFIPNetworking62109.2024.10619827
Han SWang SZhong CZhou DHuang TWei LTan C(2024)Openlab: A One-Stop Service Platform for Software-Defined Networks EmulationProceedings of the 2024 SIGCOMM Workshop on Formal Methods Aided Network Operation10.1145/3672199.3673893(45-47)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3672199.3673893
Brown LAlcoz ACangialosi FNarayan AAlizadeh MBalakrishnan HFriedman EKatz-Bassett EKrishnamurthy ASchapira MShenker SSekar VYu MSeneviratne AVeitch D(2024)Principles for Internet Congestion ManagementProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672247(166-180)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672247
Han XHan BLi JSong C(2024)Multi-agent DRL-based Multipath Scheduling for Video Streaming with QUICACM Transactions on Multimedia Computing, Communications, and Applications10.1145/364913920:7(1-23)Online publication date: 15-Mar-2024
https://dl.acm.org/doi/10.1145/3649139
Yu HWang HTian CSathyanarayana SShi SXue ZYu SLi HPeng YPang XZhang RVallina-Rodríguez NSuarez-Tángil GLevin DPelsser C(2024)Magpie: Improving the Efficiency of A/B Tests for Large Scale Video-on-Demand SystemsProceedings of the 2024 ACM on Internet Measurement Conference10.1145/3646547.3689019(588-594)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3646547.3689019
Giacomoni LParisis G(2024)Reinforcement Learning-based Congestion Control: A Systematic Evaluation of Fairness, Efficiency and ResponsivenessIEEE INFOCOM 2024 - IEEE Conference on Computer Communications10.1109/INFOCOM52122.2024.10621288(1451-1460)Online publication date: 20-May-2024
https://doi.org/10.1109/INFOCOM52122.2024.10621288
Wu ZLi HBai HPei X(2024)A Scalable and Resilient Protocol for Synchronous Collaboration2024 27th International Conference on Computer Supported Cooperative Work in Design (CSCWD)10.1109/CSCWD61410.2024.10580450(441-446)Online publication date: 8-May-2024
https://doi.org/10.1109/CSCWD61410.2024.10580450
Nguyen MLam ANguyen PTruong H(2024)Security Orchestration with Explainability for Digital Twins-Based Smart Systems2024 IEEE 48th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC61105.2024.00159(1194-1203)Online publication date: 2-Jul-2024
https://doi.org/10.1109/COMPSAC61105.2024.00159
Xie JMa CYu HLuo LSun G(2024)Analysis and Optimization for Passive One-way Delay Measurement Tax in Container Networks2024 IEEE 17th International Conference on Cloud Computing (CLOUD)10.1109/CLOUD62652.2024.00036(247-255)Online publication date: 7-Jul-2024
https://doi.org/10.1109/CLOUD62652.2024.00036
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents