research-article

Learning, indexing, and diagnosing network faults

Authors:

Mudhakar Srivatsa,

Dakshi Agrawal,

Ling LiuAuthors Info & Claims

KDD '09: Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining

Pages 857 - 866

https://doi.org/10.1145/1557019.1557113

Published: 28 June 2009 Publication History

Abstract

Modern communication networks generate massive volume of operational event data, e.g., alarm, alert, and metrics, which can be used by a network management system (NMS) to diagnose potential faults. In this work, we introduce a new class of indexable fault signatures that encode temporal evolution of events generated by a network fault as well as topological relationships among the nodes where these events occur. We present an efficient learning algorithm to extract such fault signatures from noisy historical event data, and with the help of novel space-time indexing structures, we show how to perform efficient, online signature matching. We provide results from extensive experimental studies to explore the efficacy of our approach and point out potential applications of such signatures for many different types of networks including social and information networks.

Supplementary Material

JPG File (p857-srivatsa.jpg)

Download
12.28 KB

MP4 File (p857-srivatsa.mp4)

Download
67.14 MB

References

[1]

R. Agrawal, A. Borgida, and H. Jagadish. Efficient management of transitive relationships in large data and knowledge bases. SIGMOD Rec., 18(2), 1989.

Digital Library

[2]

Y. Ahmad and S. Nath. COLR-tree: communication-efficient spatio-temporal indexing for a sensor data web portal. In ICDE, 2008.

Digital Library

[3]

H. Akaike. A new look at the statistical model identification. IEEE Trans. Auto. Cont., 19(6), 1974.

[4]

A.-L. Barabási. Linked: The New Science of Networks. Perseus Publishing, 2002.

Digital Library

[5]

E. Cohen, E. Halperin, H. Kaplan, and U. Zwick. Reachability and distance queries via 2-hop labels. SIAM J. Comput., 32(5), 2003.

Digital Library

[6]

I. Cohen, S. Zhang, M. Goldszmidt, J. Symons, T. Kelly, and A. Fox. Capturing, indexing, clustering, and retrieving system history. In SOSP, 2005.

Digital Library

[7]

A. Dempster, N. Laird, and D. Rubin. Maximum likelihood from incomplete data via the em algorithm. J. Royal Stat. Soci. B, 39(1), 1977.

[8]

L. Fan, P. Cao, J. Almeida, and A. Broder. Summary cache: A scalable wide-area web cache sharing protocol. In IEEE/ACM Trans. Netw., 1998.

Digital Library

[9]

A. Feldmann, O. Maennel, Z. Mao, A. Berger, and B. Maggs. Locating internet routing instabilities. SIGCOMM Comput. Commun. Rev., 34(4), 2004.

Digital Library

[10]

I. E. T. Force. OSPF version 2. http://www.ietf.org/rfc.

[11]

A. Guttman. R-trees: A dynamic index structure for spatial searching. In SIGMOD, 1984.

Digital Library

[12]

Y. Huang, N. Feamster, A. Lakhina, and J. Xu. Diagnosing network disruptions with network-wide analysis. SIGMETRICS Perform. Eval. Rev., 35(1), 2007.

Digital Library

[13]

I. Katzela and M. Schwartz. Schemes for fault identification in communication networks. IEEE/ACM Trans. Netw., 3(6), 1995.

Digital Library

[14]

A. Lakhina, M. Crovella, and C. Diot. Mining anomalies using traffic feature distributions. SIGCOMM Comput. Commun. Rev., 35(4), 2005.

Digital Library

[15]

L. Lewis. A case-based reasoning approach to the resolution of faults in communication networks. In IM, 1993.

Digital Library

[16]

X. Meng, G. Jiang, H. Zhang, H. Chen, and K. Yoshihira. Automatic profiling of network event sequences: algorithm and application. In IEEE INFOCOM, 2008.

[17]

Y. Nygate. Event correlation using rule and object based techniques. In IM, 1995.

Digital Library

[18]

J. Pearl. Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann Publishers Inc., 1988.

Digital Library

[19]

F. Salfner. Event-based failure prediction: an extended hidden markov model approach. Department of Computer Science, Humboldt-Universitat zu Berlin, Germany, 2008.

[20]

M. Steinder and A. Sethi. A survey of fault localization techniques in computer networks. Sci. Comput. Prog., 53, 2004.

[21]

P. Wu, R. Bhatnagar, L. Epshtein, M. Bhandaru, and S. Zhongwen. Alarm correlation engine. In NOMS, 1998.

[22]

S. Yemini, S. Kliger, E. Mozes, Y. Yemini, and D. Ohsie. High speed and robust event correlation. Communications Magazine, IEEE, 34(5), 1996.

Digital Library

[23]

C. Yuan, N. Lao, J.-R. Wen, J. Li, Z. Zhang, Y.-M. Wang, and W.-Y. Ma. Automated known problem diagnosis with event traces. In EuroSys, 2006.

Digital Library

[24]

J. Zhang, J. Rexford, and J. Feigenbaum. Learning-based anomaly detection in BGP updates. In MineNet, 2005.

Digital Library

[25]

T. Zhang, R. Ramakrishnan, and M. Livny. BIRCH: an efficient data clustering method for very large databases. In SIGMOD, 1996.

Digital Library

Cited By

He QTan SChen FXu XQi LHei XJin HYang YChen HDuh WHuang HKato MMothe JPoblete B(2023)EDIndex: Enabling Fast Data Queries in Edge Storage SystemsProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591676(675-685)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591676
Munir AMortazavi SBahnasy MBaniamerian AWang SGuan SGanjali Y(2022)SmartTagsProceedings of the ACM SIGCOMM Workshop on Network-Application Integration10.1145/3538401.3546601(46-52)Online publication date: 22-Aug-2022
https://dl.acm.org/doi/10.1145/3538401.3546601
Li JJiang YZhang Z(2021)Network root fault location based on network topology and alarm2021 2nd Asia Service Sciences and Software Engineering Conference10.1145/3456126.3456138(102-107)Online publication date: 24-Feb-2021
https://dl.acm.org/doi/10.1145/3456126.3456138
Show More Cited By

Index Terms

Learning, indexing, and diagnosing network faults
1. Information systems
  1. Information systems applications
    1. Data mining
2. Networks
  1. Network services
    1. Network monitoring

Recommendations

CORONET: Fault tolerance for Software Defined Networks
ICNP '12: Proceedings of the 2012 20th IEEE International Conference on Network Protocols (ICNP)

Software Defined Networking, or SDN, based networks are being deployed not only in testbed networks, but also in production networks. Although fault-tolerance is one of the most desirable properties in production networks, there are not much study in ...
A Novel Fault Diagnosis in Reversible Logic Circuit
HPCC '14: Proceedings of the 2014 IEEE Intl Conf on High Performance Computing and Communications, 2014 IEEE 6th Intl Symp on Cyberspace Safety and Security, 2014 IEEE 11th Intl Conf on Embedded Software and Syst (HPCC,CSS,ICESS)

Reversible circuits have widespread applications in quantum computing, low power circuit design, nanotechnology, cryptography etc. The paper addresses the fault diagnosis problem in reversible logic circuit for missing gate fault model. The key idea of ...
Network Traffic Analysis in Software-Defined Networking Using RYU Controller
Abstract
Software-Defined Networking (SDN) has emerged as a promising paradigm to enhance network control and management by decoupling the planes. With SDN, the centralized controller plays a critical role in managing network resources and traffic flows. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '09: Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining

June 2009

1426 pages

ISBN:9781605584959

DOI:10.1145/1557019

General Chairs:
John Elder
Elder Research, Inc., USA
,
Françoise Soulié Fogelman
KXEN, France
,
Program Chairs:
Peter Flach
University of Bristol, UK
,
Mohammed Zaki
RPI, USA

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 June 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

KDD09

Sponsor:

KDD09: The 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

June 28 - July 1, 2009

Paris, France

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
647
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

He QTan SChen FXu XQi LHei XJin HYang YChen HDuh WHuang HKato MMothe JPoblete B(2023)EDIndex: Enabling Fast Data Queries in Edge Storage SystemsProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591676(675-685)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591676
Munir AMortazavi SBahnasy MBaniamerian AWang SGuan SGanjali Y(2022)SmartTagsProceedings of the ACM SIGCOMM Workshop on Network-Application Integration10.1145/3538401.3546601(46-52)Online publication date: 22-Aug-2022
https://dl.acm.org/doi/10.1145/3538401.3546601
Li JJiang YZhang Z(2021)Network root fault location based on network topology and alarm2021 2nd Asia Service Sciences and Software Engineering Conference10.1145/3456126.3456138(102-107)Online publication date: 24-Feb-2021
https://dl.acm.org/doi/10.1145/3456126.3456138
KIMURA TWATANABE ATOYONO TISHIBASHI K(2019)Proactive Failure Detection Learning Generation Patterns of Large-Scale Network LogsIEICE Transactions on Communications10.1587/transcom.2018EBP3103E102.B:2(306-316)Online publication date: 1-Feb-2019
https://doi.org/10.1587/transcom.2018EBP3103
Meng FXu JZhang XYang LChen PWang YLiu XAyachitula NMurthy KShwartz LGalambos GSu ZZheng J(2018)Opportunities and Challenges Towards Cognitive IT Service Management in Real World2018 IEEE Symposium on Service-Oriented System Engineering (SOSE)10.1109/SOSE.2018.00028(164-173)Online publication date: Mar-2018
https://doi.org/10.1109/SOSE.2018.00028
KIMURA TISHIBASHI KMORI TSAWADA HTOYONO TNISHIMATSU KWATANABE ASHIMODA ASHIOMOTO K(2017)Network Event Extraction from Log Data with Nonnegative Tensor FactorizationIEICE Transactions on Communications10.1587/transcom.2016EBP3430E100.B:10(1865-1878)Online publication date: 2017
https://doi.org/10.1587/transcom.2016EBP3430
(2017)Efficient alarm behavior analytics for telecom networksInformation Sciences: an International Journal10.1016/j.ins.2017.03.020402:C(1-14)Online publication date: 1-Sep-2017
https://dl.acm.org/doi/10.1016/j.ins.2017.03.020
Nguyen HScalosub GZheng R(2014)On Quality of Monitoring for Multichannel Wireless Infrastructure NetworksIEEE Transactions on Mobile Computing10.1109/TMC.2013.2313:3(664-677)Online publication date: 1-Mar-2014
https://dl.acm.org/doi/10.1109/TMC.2013.23
Kimura TIshibashi KMori TSawada HToyono TNishimatsu KWatanabe AShimoda AShiomoto K(2014)Spatio-temporal factorization of log data for understanding network eventsIEEE INFOCOM 2014 - IEEE Conference on Computer Communications10.1109/INFOCOM.2014.6847986(610-618)Online publication date: Apr-2014
https://doi.org/10.1109/INFOCOM.2014.6847986
Zong BRaghavendra RSrivatsa MYan XSingh ALee K(2014)Cloud service placement via subgraph matching2014 IEEE 30th International Conference on Data Engineering10.1109/ICDE.2014.6816704(832-843)Online publication date: Mar-2014
https://doi.org/10.1109/ICDE.2014.6816704
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents