short-paper

Network-Wide Heavy Hitter Detection with Commodity Switches

Authors:

Jennifer RexfordAuthors Info & Claims

SOSR '18: Proceedings of the Symposium on SDN Research

Article No.: 8, Pages 1 - 7

https://doi.org/10.1145/3185467.3185476

Published: 28 March 2018 Publication History

Abstract

Many network monitoring tasks identify subsets of traffic that stand out, e.g., top-k flows for a particular statistic. A Protocol Independent Switch Architecture (PISA) switch can identify these "heavy hitter" flows directly in the data plane, by aggregating traffic statistics across packets and comparing against a threshold. However, network operators often want to identify interesting traffic on a network-wide basis. To bridge the gap between line-rate monitoring and network-wide visibility, we present a distributed heavy-hitter detection scheme for networks modeled as one-big switch. We use adaptive thresholds to perform efficient threshold monitoring directly in the data plane. We implement our system using the P4 language, and evaluate it using real-world packet traces. We demonstrate that our solution can accurately detect network-wide heavy hitters with up to 70% savings in communication overhead compared to an existing approach with a provable upper bound.

References

[1]

Brian Babcock and Chris Olston. 2003. Distributed Top-k Monitoring. In ACM SIGMOD International Conference on Management of Data. ACM, 28--39.

Digital Library

[2]

Pat Bosshart, Dan Daly, Glen Gibb, Martin Izzard, Nick McKeown, Jennifer Rexford, Cole Schlesinger, Dan Talayco, Amin Vahdat, George Varghese, and David Walker. 2014. P4: Programming Protocol-independent Packet Processors. ACM SIGCOMM Computer Communication Review 44, 3 (2014), 87--95.

Digital Library

[3]

Pat Bosshart, Glen Gibb, Hun-Seok Kim, George Varghese, Nick McKeown, Martin Izzard, Fernando Mujica, and Mark Horowitz. 2013. Forwarding Metamorphosis: Fast Programmable Match-action Processing in Hardware for SDN. In ACM SIGCOMM. ACM, New York, NY, USA, 99--110.

Digital Library

[4]

Yanpei Chen, Rean Griffith, Junda Liu, Randy H. Katz, and Anthony D. Joseph. 2009. Understanding TCP Incast Throughput Collapse in Datacenter Networks. In ACM SIGCOMM Workshop on Research on Enterprise Networking. ACM, New York, NY, USA, 73--82.

Digital Library

[5]

Benoit Claise. 2004. Cisco Systems NetFlow Services Export Version 9. RFC 3954. RFC Editor. http://www.rfc-editor.org/rfc/rfc3954.txt

[6]

Benoit Claise. 2008. Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of IP Traffic Flow Information. RFC 5101. RFC Editor. http://www.rfc-editor.org/rfc/rfc5101.txt

[7]

Graham Cormode. 2011. Continuous Distributed Monitoring: A Short Survey. In International Workshop on Algorithms and Models for Distributed Event Processing. ACM, 1--10.

Digital Library

[8]

Graham Cormode and Shan Muthukrishnan. 2005. An improved data stream summary: the count-min sketch and its applications. Journal of Algorithms 55, 1 (2005), 58--75.

Digital Library

[9]

Graham Cormode, S Muthukrishnan, and Ke Yi. 2011. Algorithms for distributed functional monitoring. ACM Transactions on Algorithms (TALG) 7, 2 (2011), 21.

Digital Library

[10]

Graham Cormode, S Muthukrishnan, Ke Yi, and Qin Zhang. 2010. Optimal Sampling From Distributed Streams. In ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems. ACM, 77--86.

Digital Library

[11]

Philippe Flajolet and G Nigel Martin. 1985. Probabilistic counting algorithms for data base applications. J. Comput. System Sci. 31, 2 (1985), 182--209.

Digital Library

[12]

Center for Applied Internet Data Analysis. 2018 (accessed November 1, 2017). The CAIDA UCSD Anonymized Internet Traces 2016. http://www.caida.org/data/passive/passive_2016_dataset.xml

[13]

Arpit Gupta, Rob Harrison, Ankita Pawar, Rüdiger Birkner, Marco Canini, Nick Feamster, Jennifer Rexford, and Walter Willinger. 2017. Sonata: Query-Driven Network Telemetry. arXiv preprint arXiv:1705.01049 (2017).

[14]

Ankur Jain, Joseph M Hellerstein, Sylvia Ratnasamy, and David Wetherall. 2004. A Wakeup Call for Internet Monitoring Systems: The Case for Distributed Triggers. In HotNets-III.

[15]

Jaeyeon Jung, Vern Paxson, Arthur W Berger, and Hari Balakrishnan. 2004. Fast portscan detection using sequential hypothesis testing. In IEEE Symposium on Security and Privacy. IEEE, 211--225.

[16]

Ram Keralapura, Graham Cormode, and Jeyashankher Ramamirtham. 2006. Communication-efficient distributed monitoring of thresholded counts. In ACM SIGMOD International Conference on Management of Data. ACM, 289--300.

Digital Library

[17]

Yuliang Li, Rui Miao, Changhoon Kim, and Minlan Yu. 2016. FlowRadar: A Better NetFlow for Data Centers. In Usenix NSDI. 311--324.

Digital Library

[18]

Zaoxing Liu, Antonis Manousis, Gregory Vorsanger, Vyas Sekar, and Vladimir Braverman. 2016. One Sketch to Rule Them All: Rethinking Network Flow Monitoring with UnivMon. In ACM SIGCOMM. ACM, 101--114.

Digital Library

[19]

Srinivas Narayana, Mina Tahmasbi, Jennifer Rexford, and David Walker. 2016. Compiling Path Queries. In Usenix NSDI. 207--222.

Digital Library

[20]

Barefoot Networks. 2018 (accessed November 1, 2017). Barefoot Tofino. https://www.barefootnetworks.com/products/brief-tofino/

[21]

Peter Phaal and Sonia Panchen. 2003 (accessed February 14, 2018). Packet Sampling Basics. http://www.sflow.org/packetSamplingBasics/index.htm

[22]

Barath Raghavan, Kashi Vishwanath, Sriram Ramabhadran, Kenneth Yocum, and Alex C. Snoeren. 2007. Cloud Control with Distributed Rate Limiting. In ACM SIGCOMM. ACM, New York, NY, USA, 337--348.

Digital Library

[23]

Brandon Schlinker, Hyojeong Kim, Timothy Cui, Ethan Katz-Bassett, Harsha V Madhyastha, Italo Cunha, James Quinn, Saif Hasan, Petr Lapukhov, and Hongyi Zeng. 2017. Engineering Egress with Edge Fabric: Steering Oceans of Content to the World. In ACM SIGCOMM. ACM, 418--431.

Digital Library

[24]

Vibhaalakshmi Sivaraman, Srinivas Narayana, Ori Rottenstreich, S Muthukrishnan, and Jennifer Rexford. 2017. Heavy-Hitter Detection Entirely in the Data Plane. In ACM SOSR. ACM, 164--176.

Digital Library

[25]

Cisco Systems. 2017 (accessed February 25, 2018). Cisco Nexus 3600 NX-OS System Management Configuration Guide. https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus3000/sw/system_mgmt/503_U4_1/b_3k_System_Mgmt_Config_503_u4_1/b_3k_System_Mgmt_Config_503_u4_1_chapter_010010.pdf

[26]

Ke Yi and Qin Zhang. 2013. Optimal tracking of distributed heavy hitters and quantiles. Algorithmica 65, 1 (2013), 206--223.

Digital Library

[27]

Minlan Yu, Lavanya Jose, and Rui Miao. 2013. Software Defined Traffic Measurement with OpenSketch. In Usenix NSDI, Vol. 13. 29--42.

Digital Library

Cited By

Gjeci FFilippini ICapone A(2024)An Optimized Handover Management Scheme Tailored for Heavy Hitters in a Disaggregated 5G O-RAN Architecture2024 IFIP Networking Conference (IFIP Networking)10.23919/IFIPNetworking62109.2024.10619753(647-653)Online publication date: 3-Jun-2024
https://doi.org/10.23919/IFIPNetworking62109.2024.10619753
Ben Basat REinziger GHan WTayh B(2024)SQUID: Faster Analytics via Sampled Quantile EstimationProceedings of the ACM on Networking10.1145/36768732:CoNEXT3(1-23)Online publication date: 21-Aug-2024
https://dl.acm.org/doi/10.1145/3676873
Li EWu WZhaohua WLi ZNiu J(2024)SAROS: A Self-Adaptive Routing Oblivious Sampling Method for Network-wide Heavy Hitter DetectionProceedings of the 8th Asia-Pacific Workshop on Networking10.1145/3663408.3663429(142-148)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.1145/3663408.3663429
Show More Cited By

Network-Wide Heavy Hitter Detection with Commodity Switches
1. Networks
  1. Network services

Recommendations

Heavy-Hitter Detection Entirely in the Data Plane
SOSR '17: Proceedings of the Symposium on SDN Research

Identifying the "heavy hitter" flows or flows with large traffic volumes in the data plane is important for several applications e.g., flow-size aware routing, DoS detection, and traffic engineering. However, measurement in the data plane is constrained ...
Heavy Hitter Detection on Multi-Pipeline Switches
ANCS '21: Proceedings of the Symposium on Architectures for Networking and Communications Systems

Recently, several applications have been designed and implemented to run entirely in the dataplane. However, most if not all the applications assume that network traffic traverses the same pipe, from ingress to egress inside the switch. While this seems ...
A cloud-scale per-flow backpressure system via FPGA-based heavy hitter detection
SIGCOMM '21: Proceedings of the SIGCOMM '21 Poster and Demo Sessions

Virtual private clouds provide sharing resources to a massive number of tenants for economics of scale. In such clouds, off-the-shelf x86 boxes are widely deployed as network intermediate nodes. However, due to rapid growth of cloud traffic and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SOSR '18: Proceedings of the Symposium on SDN Research

March 2018

195 pages

ISBN:9781450356640

DOI:10.1145/3185467

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCOMM: ACM Special Interest Group on Data Communication
ONS: Open Networking Summit

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 March 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Short-paper
Research
Refereed limited

Conference

SOSR '18

Sponsor:

SIGCOMM
ONS

SOSR '18: Symposium on SDN Research

March 28 - 29, 2018

CA, Los Angeles, USA

Acceptance Rates

Overall Acceptance Rate 7 of 43 submissions, 16%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

79
Total Citations
View Citations
991
Total Downloads

Downloads (Last 12 months)81
Downloads (Last 6 weeks)9

Reflects downloads up to 06 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gjeci FFilippini ICapone A(2024)An Optimized Handover Management Scheme Tailored for Heavy Hitters in a Disaggregated 5G O-RAN Architecture2024 IFIP Networking Conference (IFIP Networking)10.23919/IFIPNetworking62109.2024.10619753(647-653)Online publication date: 3-Jun-2024
https://doi.org/10.23919/IFIPNetworking62109.2024.10619753
Ben Basat REinziger GHan WTayh B(2024)SQUID: Faster Analytics via Sampled Quantile EstimationProceedings of the ACM on Networking10.1145/36768732:CoNEXT3(1-23)Online publication date: 21-Aug-2024
https://dl.acm.org/doi/10.1145/3676873
Li EWu WZhaohua WLi ZNiu J(2024)SAROS: A Self-Adaptive Routing Oblivious Sampling Method for Network-wide Heavy Hitter DetectionProceedings of the 8th Asia-Pacific Workshop on Networking10.1145/3663408.3663429(142-148)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.1145/3663408.3663429
Lu JPan THe SMiao MZhou GQi YZhang SSong ESun XZhao HLyu BZhu S(2024)CloudSentry: Two-Stage Heavy Hitter Detection for Cloud-Scale Gateway Overload ProtectionIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2023.330185235:4(616-633)Online publication date: Apr-2024
https://doi.org/10.1109/TPDS.2023.3301852
Sun HHuang QLee PBai WZhu FBao Y(2024)Distributed Network Telemetry With Resource Efficiency and Full AccuracyIEEE/ACM Transactions on Networking10.1109/TNET.2023.332734532:3(1857-1872)Online publication date: Jun-2024
https://doi.org/10.1109/TNET.2023.3327345
Gu JSong CDai HLu LLiu M(2024)Compact Estimator for Streaming Triangle CountingIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.337122836:8(3712-3724)Online publication date: Aug-2024
https://doi.org/10.1109/TKDE.2024.3371228
Silva Rodrigues TLuciano Verdi F(2024)Detecting Heavy Hitters in Network-Wide Programmable Multi-Pipe DevicesNOMS 2024-2024 IEEE Network Operations and Management Symposium10.1109/NOMS59830.2024.10575503(1-5)Online publication date: 6-May-2024
https://doi.org/10.1109/NOMS59830.2024.10575503
Ding D(2024)CARBINE: Exploring Additional Properties of HyperLogLog for Secure and Robust Flow Cardinality EstimationIEEE INFOCOM 2024 - IEEE Conference on Computer Communications10.1109/INFOCOM52122.2024.10621185(471-480)Online publication date: 20-May-2024
https://doi.org/10.1109/INFOCOM52122.2024.10621185
Graf JChuprikov PEugster PJahnke P(2024)FARM: Comprehensive Data Center Network Monitoring and Management2024 IEEE 44th International Conference on Distributed Computing Systems (ICDCS)10.1109/ICDCS60910.2024.00055(520-530)Online publication date: 23-Jul-2024
https://doi.org/10.1109/ICDCS60910.2024.00055
Ding DKesgin OZilberman N(2024)INDDoS+: Secure DDoS Detection Mechanism in Programmable Switches2024 IEEE 25th International Conference on High Performance Switching and Routing (HPSR)10.1109/HPSR62440.2024.10635967(197-202)Online publication date: 22-Jul-2024
https://doi.org/10.1109/HPSR62440.2024.10635967
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents