research-article

Free access

Compressing IP forwarding tables: towards entropy bounds and beyond

Authors:

Gábor Rétvári,

János Tapolcai,

Attila Kőrösi,

András Majdán,

Zalán HeszbergerAuthors Info & Claims

SIGCOMM '13: Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM

Pages 111 - 122

https://doi.org/10.1145/2486001.2486009

Published: 27 August 2013 Publication History

Abstract

Lately, there has been an upsurge of interest in compressed data structures, aiming to pack ever larger quantities of information into constrained memory without sacrificing the efficiency of standard operations, like random access, search, or update. The main goal of this paper is to demonstrate how data compression can benefit the networking community, by showing how to squeeze the IP Forwarding Information Base (FIB), the giant table consulted by IP routers to make forwarding decisions, into information-theoretical entropy bounds, with essentially zero cost on longest prefix match and FIB update. First, we adopt the state-of-the-art in compressed data structures, yielding a static entropy-compressed FIB representation with asymptotically optimal lookup. Then, we re-design the venerable prefix tree, used commonly for IP lookup for at least 20 years in IP routers, to also admit entropy bounds and support lookup in optimal time and update in nearly optimal time. Evaluations on a Linux kernel prototype indicate that our compressors encode a FIB comprising more than 440K prefixes to just about 100--400 KBytes of memory, with a threefold increase in lookup throughput and no penalty on FIB updates.

References

[1]

M. Bando, Y.-L. Lin, and H. J. Chao. FlashTrie: beyond 100-Gb/s IP route lookup using hash-based prefix-compressed trie. IEEE/ACM Trans. Netw., 20(4):1262--1275, 2012.

Digital Library

[2]

R. Bolla and R. Bruschi. RFC 2544 performance evaluation and internal measurements for a Linux based open router. In IEEE HPSR, page 6, 2006.

[3]

J. Bonwick and B. Moore. ZFS - the last word in file systems. Sun Microsystems, 2004.

[4]

R. E. Bryant. Symbolic boolean manipulation with ordered binary-decision diagrams. ACM Comput. Surv., 24(3):293--318, 1992.

Digital Library

[5]

G. Cheung and S. McCanne. Optimal routing table design for IP address lookups under memory constraints. In IEEE INFOCOM, pages 1437--1444, 1999.

[6]

J. Choi, J. H. Park, P. chun Cheng, D. Kim, and L. Zhang. Understanding BGP next-hop diversity. In INFOCOM Workshops, pages 846 --851, 2011.

[7]

J. Cocke. Global common subexpression elimination. SIGPLAN Not., 5(7):20--24, 1970.

Digital Library

[8]

T. M. Cover and J. A. Thomas. Elements of information theory. Wiley-Interscience, 1991.

Digital Library

[9]

E. de Moura, G. Navarro, N. Ziviani, and R. Baeza-Yates. Fast and flexible word searching on compressed text. ACM Trans. Inf. Syst., 18(2):113--139, 2000.

Digital Library

[10]

M. Degermark, A. Brodnik, S. Carlsson, and S. Pink. Small forwarding tables for fast routing lookups. In ACM SIGCOMM, pages 3--14, 1997.

Digital Library

[11]

S. Dharmapurikar, P. Krishnamurthy, and D. E. Taylor. Longest prefix matching using Bloom filters. In ACM SIGCOMM, pages 201--212, 2003.

Digital Library

[12]

R. Draves, C. King, S. Venkatachary, and B. Zill. Constructing optimal IP routing tables. In IEEE INFOCOM, 1999.

[13]

W. Eatherton, G. Varghese, and Z. Dittia. Tree bitmap: hardware/software IP lookups with incremental updates. SIGCOMM Comput. Commun. Rev., 34(2):97--122, 2004.

Digital Library

[14]

A. Elmokashfi, A. Kvalbein, and C. Dovrolis. BGP churn evolution: a perspective from the core. IEEE/ACM Trans. Netw., 20(2):571--584, 2012.

Digital Library

[15]

EZChip. NP-4: 100-Gigabit Network Processor for Carrier Ethernet Applications. http://www.ezchip.com/Images/pdf/NP-4_Short_Brief_online.pdf, 2011.

[16]

K. Fall, G. Iannaccone, S. Ratnasamy, and P. B. Godfrey. Routing tables: Is smaller really much better? In ACM HotNets-VIII, 2009.

[17]

P. Ferragina, F. Luccio, G. Manzini, and S. Muthukrishnan. Compressing and indexing labeled trees, with applications. J. ACM, 57(1):1--33, 2009.

Digital Library

[18]

P. Ferragina and G. Manzini. Opportunistic data structures with applications. In IEEE FOCS, pages 390--398, 2000.

Digital Library

[19]

P. Ferragina, G. Manzini, V. M\"akinen, and G. Navarro. Compressed representations of sequences and full-text indexes. ACM Trans. Algorithms, 3(2), 2007.

Digital Library

[20]

P. Francois, C. Filsfils, J. Evans, and O. Bonaventure. Achieving sub-second IGP convergence in large IP networks. SIGCOMM Comput. Commun. Rev., 35(3):35--44, 2005.

Digital Library

[21]

P. Gupta, B. Prabhakar, and S. P. Boyd. Near optimal routing lookups with bounded worst case performance. In IEEE INFOCOM, pages 1184--1192, 2000.

[22]

S. Han, K. Jang, K. Park, and S. Moon. PacketShader: a GPU-accelerated software router. In ACM SIGCOMM, pages 195--206, 2010.

Digital Library

[23]

J. Hasan and T. N. Vijaykumar. Dynamic pipelining: making IP-lookup truly scalable. In ACM SIGCOMM, pages 205--216, 2005.

Digital Library

[24]

P. Hick, kc claffy, and D. Andersen. CAIDA Anonymized Internet Traces. http://www.caida.org/data/passive.

[25]

W.-K. Hon, R. Shah, and J. S. Vitter. Compression, indexing, and retrieval for massive string data. In CPM, pages 260--274, 2010.

Digital Library

[26]

G. Huston. BGP routing table analysis reports. http://bgp.potaroo.net/.

[27]

I. Ioannidis and A. Grama. Level compressed DAGs for lookup tables. Comput. Netw., 49(2):147--160, 2005.

Digital Library

[28]

G. Jacobson. Space-efficient static trees and graphs. In IEEE FOCS, pages 549--554, 1989.

Digital Library

[29]

J. Katajainen and E. M\"akinen. Tree compression and optimization with applications. International Journal of Foundations of Computer Science, 1(4):425--447, 1990.

[30]

V. Khare, D. Jen, X. Zhao, Y. Liu, D. Massey, L. Wang, B. Zhang, and L. Zhang. Evolution towards global routing scalability. IEEE JSAC, 28(8):1363--1375, 2010.

Digital Library

[31]

Y. Liu, S. O. Amin, and L. Wang. Efficient FIB caching using minimal non-overlapping prefixes. SIGCOMM Comput. Commun. Rev., 43(1):14--21, Jan. 2012.

Digital Library

[32]

LuceneTransform. Transparent compression for Apache Lucene. http://code.google.com/p/lucenetransform.

[33]

H. Madhyastha, E. Katz-Bassett, T. Anderson, A. Krishnamurthy, and A. Venkataramani. iPlane Nano: path prediction for peer-to-peer applications. In USENIX, pages 137--152, 2009.

Digital Library

[34]

V. M\"akinen and G. Navarro. Dynamic entropy compressed sequences and full-text indexes. ACM Trans. Algorithms, 4(3):32:1--32:38, 2008.

Digital Library

[35]

A. McAuley and P. Francis. Fast routing table lookup using CAMs. In IEEE INFOCOM, pages 1382--1391, 1993.

[36]

D. Meyer, L. Zhang, and K. Fall. Report from the IAB Workshop on Routing and Addressing. RFC 4984, 2007.

[37]

D. S. Miller. net\_test\_tools. https://kernel.googlesource.com/pub/scm/linux/kernel/git/davem/net_test_tools.

[38]

G. Navarro and F. Claude. libcds: Compact data structures library, 2004. http://libcds.recoded.cl.

[39]

G. Navarro and V. M\"akinen. Compressed full-text indexes. ACM Comput. Surv., 39(1), 2007.

Digital Library

[40]

G. Navarro and J. Tarhio. LZgrep: a Boyer-Moore string matching tool for Ziv-Lempel compressed text. Softw. Pract. Exper., 35(12):1107--1130, 2005.

Digital Library

[41]

S. Nilsson and G. Karlsson. IP-address lookup using LC-tries. IEEE JSAC, 17(6):1083 --1092, 1999.

Digital Library

[42]

R. Raman, V. Raman, and S. S. Rao. Succinct indexable dictionaries with applications to encoding k-ary trees and multisets. In ACM-SIAM SODA, pages 233--242, 2002.

Digital Library

[43]

G. Rétvári, Z. Csernátony, A. Körösi, J. Tapolcai, A. Császár, G. Enyedi, and G. Pongrácz. Compressing IP forwarding tables for fun and profit. In ACM HotNets-XI, pages 1--6, 2012.

Digital Library

[44]

J. Rossignac. Edgebreaker: Connectivity compression for triangle meshes. IEEE Trans. Visual Comput. Graphics, 5:47--61, 1999.

Digital Library

[45]

K. Sklower. A tree-based packet routing table for Berkeley UNIX. Technical Report, Berkeley, 1991.

[46]

H. Song, M. S. Kodialam, F. Hao, and T. V. Lakshman. Scalable IP lookups using Shape Graphs. In IEEE ICNP, pages 73--82, 2009.

Digital Library

[47]

H. Song, J. Turner, and J. Lockwood. Shape shifting tries for faster IP route lookup. In IEEE ICNP, pages 358--367, 2005.

Digital Library

[48]

V. Srinivasan and G. Varghese. Faster IP lookups using controlled prefix expansion. SIGMETRICS Perform. Eval. Rev., 26(1):1--10, 1998.

Digital Library

[49]

S. Stergiou and J. Jain. Optimizing routing tables on systems-on-chip with content-addressable memories. In System-on-Chip, pages 1--6, 2008.

[50]

H. Sun, O. Ozturk, and H. Ferhatosmanoglu. CoMRI: a compressed multi-resolution index structure for sequence similarity queries. In IEEE CSB, pages 553--, 2003.

Digital Library

[51]

R. Teixeira, K. Marzullo, S. Savage, and G. M. Voelker. In search of path diversity in ISP networks. In ACM IMC, pages 313--318, 2003.

Digital Library

[52]

P. M. Tolani and J. R. Haritsa. XGRIND: a query-friendly XML compressor. In ICDE, pages 225--234, 2002.

Digital Library

[53]

Z. A. Uzmi, M. Nebel, A. Tariq, S. Jawad, R. Chen, A. Shaikh, J. Wang, and P. Francis. SMALTA: practical and near-optimal FIB aggregation. In ACM CoNEXT, pages 1--12, 2011.

Digital Library

[54]

S. Vigna and P. Boldi. MG4J: Managing Gigabytes for Java. http://mg4j.dsi.unimi.it, 2007.

[55]

M. Waldvogel, G. Varghese, J. Turner, and B. Plattner. Scalable high speed IP routing lookups. In ACM SIGCOMM, pages 25--36, 1997.

Digital Library

[56]

WebGraph. A framework for graph compression. http://webgraph.di.unimi.it.

[57]

I. H. Witten, A. Moffat, and T. C. Bell. Managing Gigabytes: Compressing and Indexing Documents and Images. Morgan Kaufmann, 1999.

Digital Library

[58]

J. Woods. PPP Deflate Protocol. RFC 1979, 1996.

Digital Library

[59]

W. Wu. Packet Forwarding Technologies. Auerbach, 2008.

[60]

M. Zec, L. Rizzo, and M. Mikuc. DXR: towards a billion routing lookups per second in software. SIGCOMM Comput. Commun. Rev., 42(5):29--36, 2012.

Digital Library

[61]

X. Zhao, D. J. Pacella, and J. Schiller. Routing scalability: an operator's view. IEEE JSAC, 28(8):1262--1270, 2010.

Digital Library

[62]

N. Ziviani, E. S. de Moura, G. Navarro, and R. Baeza-Yates. Compression: A key for next-generation text retrieval systems. IEEE Computer, 33(11):37--44, 2000.

Digital Library

Cited By

Ficzere DHollósi GFrankó AVarga PBiró J(2024)Orderliness of Navigation Patterns in Hyperbolic Complex NetworksComplex Networks & Their Applications XII10.1007/978-3-031-53472-0_23(271-282)Online publication date: 21-Feb-2024
https://doi.org/10.1007/978-3-031-53472-0_23
Michel OBifulco RRétvári GSchmid S(2021)The Programmable Data PlaneACM Computing Surveys10.1145/344786854:4(1-36)Online publication date: 3-May-2021
https://dl.acm.org/doi/10.1145/3447868
Hemalatha MRukmanidevi SShanker N(2020)Searching time operation reduced IPV6 matching through dynamic DNA routing table for less memory and fast IP processingSoft Computing10.1007/s00500-020-05379-5Online publication date: 31-Oct-2020
https://doi.org/10.1007/s00500-020-05379-5
Show More Cited By

Index Terms

Compressing IP forwarding tables: towards entropy bounds and beyond
1. Information systems
  1. Data management systems
    1. Data structures
      1. Data layout
        Data compression
2. Networks
  1. Network types

Recommendations

Compressing IP forwarding tables: towards entropy bounds and beyond

Lately, there has been an upsurge of interest in compressed data structures, aiming to pack ever larger quantities of information into constrained memory without sacrificing the efficiency of standard operations, like random access, search, or update. ...
Compressing IP forwarding tables: towards entropy bounds and beyond

Lately, there has been an upsurge of interest in compressed data structures, aiming to pack ever larger quantities of information into constrained memory without sacrificing the efficiency of standard operations, like random access, search, or update. ...
Compressing IP Forwarding Tables: Realizing Information-Theoretical Space Bounds and Fast Lookups Simultaneously
ICNP '14: Proceedings of the 2014 IEEE 22nd International Conference on Network Protocols

The Internet routing ecosystem is facing compelling scalability challenges, manifested primarily in the rapid growth of IP packet forwarding tables. The forwarding table, implemented at the data plane fast path of Internet routers to drive the packet ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGCOMM '13: Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM

August 2013

580 pages

ISBN:9781450320566

DOI:10.1145/2486001

General Chairs:
Dah Ming Chiu
Chinese University of Hong Kong, China
,
Jia Wang
AT&T Labs - Research, USA
,
Program Chairs:
Paul Barford
University of Wisconsin, USA
,
Srinivasan Seshan
Carnegie Mellon University, USA

ACM SIGCOMM Computer Communication Review Volume 43, Issue 4
October 2013
595 pages
ISSN:0146-4833
DOI:10.1145/2534169
Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCOMM: ACM Special Interest Group on Data Communication

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 August 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGCOMM'13

Sponsor:

SIGCOMM

SIGCOMM'13: ACM SIGCOMM 2013 Conference

August 12 - 16, 2013

Hong Kong, China

Acceptance Rates

SIGCOMM '13 Paper Acceptance Rate 38 of 246 submissions, 15%;

Overall Acceptance Rate 462 of 3,389 submissions, 14%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

84
Total Citations
View Citations
801
Total Downloads

Downloads (Last 12 months)108
Downloads (Last 6 weeks)21

Reflects downloads up to 05 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ficzere DHollósi GFrankó AVarga PBiró J(2024)Orderliness of Navigation Patterns in Hyperbolic Complex NetworksComplex Networks & Their Applications XII10.1007/978-3-031-53472-0_23(271-282)Online publication date: 21-Feb-2024
https://doi.org/10.1007/978-3-031-53472-0_23
Michel OBifulco RRétvári GSchmid S(2021)The Programmable Data PlaneACM Computing Surveys10.1145/344786854:4(1-36)Online publication date: 3-May-2021
https://dl.acm.org/doi/10.1145/3447868
Hemalatha MRukmanidevi SShanker N(2020)Searching time operation reduced IPV6 matching through dynamic DNA routing table for less memory and fast IP processingSoft Computing10.1007/s00500-020-05379-5Online publication date: 31-Oct-2020
https://doi.org/10.1007/s00500-020-05379-5
Pomonis MPetsios TKeromytis APolychronakis MKemerlis V(2019)Kernel Protection Against Just-In-Time Code ReuseACM Transactions on Privacy and Security10.1145/327759222:1(1-28)Online publication date: 4-Jan-2019
https://dl.acm.org/doi/10.1145/3277592
Lucchesi ADrummond ATeodoro G(2018)High-performance IP lookup using Intel Xeon Phi: a Bloom filters based approachJournal of Internet Services and Applications10.1186/s13174-017-0075-y9:1Online publication date: 19-Feb-2018
https://doi.org/10.1186/s13174-017-0075-y
Wang PMcHale LGratz PSprintson A(2018)GenMatcherACM Transactions on Architecture and Code Optimization10.1145/328166315:4(1-22)Online publication date: 16-Nov-2018
https://dl.acm.org/doi/10.1145/3281663
Willison RLowry P(2018)Disentangling the Motivations for Organizational Insider Computer Abuse through the Rational Choice and Life Course PerspectivesACM SIGMIS Database: the DATABASE for Advances in Information Systems10.1145/3210530.321053749:SI(81-102)Online publication date: 25-Apr-2018
https://dl.acm.org/doi/10.1145/3210530.3210537
Pan CXie MLiu YWang YXue CWang YChen YHu J(2017)A lightweight progress maximization scheduler for non-volatile processor under unstable energy harvestingACM SIGPLAN Notices10.1145/3140582.308103852:5(101-110)Online publication date: 21-Jun-2017
https://dl.acm.org/doi/10.1145/3140582.3081038
Ding XZhang ZJia ZJu LZhao MHuang H(2017)Unified nvTCAM and sTCAM architecture for improving packet matching performanceACM SIGPLAN Notices10.1145/3140582.308103452:5(91-100)Online publication date: 21-Jun-2017
https://dl.acm.org/doi/10.1145/3140582.3081034
Zhang WFan RWen YLiu F(2017)Energy-Efficient Mobile Video StreamingACM Transactions on Intelligent Systems and Technology10.1145/31023019:1(1-16)Online publication date: 21-Aug-2017
https://dl.acm.org/doi/10.1145/3102301
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten