article

Wrapper verification

Author:

Nicholas KushmerickAuthors Info & Claims

World Wide Web, Volume 3, Issue 2

Pages 79 - 94

https://doi.org/10.1023/A:1019229612909

Published: 15 March 2000 Publication History

Abstract

Many Internet information-management applications (e.g., information integration systems) require a library of wrappers, specialized information extraction procedures that translate a source's native format into a structured representation suitable for further application-specific processing. Maintaining wrappers is tedious and error-prone, because the formatting regularities on which wrappers rely change frequently on the decentralized and dynamic Internet. The wrapper verification problem is to determine whether a wrapper is operating correctly. Standard regression testing approaches are inappropriate, because both the formatting regularities on which wrappers rely and the source's underlying content may change. We introduce RAPTURE, a fully-implemented, domain-independent wrapper verification algorithm. RAPTURE computes a probabilistic similarity measure between a wrapper's expected and observed output, where similarity is defined in terms of simple numeric features (e.g., the length, or the fraction of punctuation characters) of the extracted strings. Experiments with numerous actual Internet sources demostrate that RAPTURE performs substantially better than standard regression testing.

References

[1]

Beizer, B. (1995), Black-Box Testing , Wiley, New York.

[2]

Cohen, W. (1999), "Recognizing Structure in Web Pages Using Similarity Querries," In Proc. 16th Nat. Conf. AI , pp. 59-66.

[3]

Cowie, J. and W. Lehnert (1996), "Information Extraction," Comm. of the ACM 39 , 1, 80-91.

Digital Library

[4]

Embley, D., D. Campbell, Y. Jiang, Y.-K. Ng, R. Smith, S. Liddle, and D. Quass (1998), "A Conceptual-Modeling Approach to Extracting Data from the Web," In Proc. Int. Conf. Conceptual Modeling , pp. 78-91.

[5]

Friedman, N. and M. Goldszmidt (1996), "Learning Bayesian Networks with Local Structure," In Proc. 12th Conf. Uncertainty in Artificial Intelligence , pp. 252-262.

[6]

Gruser, J.-B., L. Raschid, M. Vidal, and L. Bright (1998), "Wrapper Generation for Web Accessible Data Sources," In Proc. Conf. Cooperative Information Systems , pp. 14-23.

[7]

Hammer, J., H. Garcia-Molina, J. Cho, R. Aranha, and A. Crespo (1997), "Extracting Semistructured Information from the Web," In Proc. Workshop on Management of Semistructured Data .

[8]

Hsu, C. and M. Dung (1998), "Generating Finite-state Transducers for Semistructured Data Extraction from the Web," J. Information Systems 23 , 8, 521-538.

Digital Library

[9]

Huck, G., P. Frankhausewr, K. Aberer, and E. Neuhold (1998), "Jedi: Extracting and Synthesizing Information from the Web," In Proc. Conf. Cooperative Information Systems , pp. 32-43.

[10]

Knoblock, A., A. Levy, O. Duschka, D. Florescu, and N. Kushmerick, Eds. (1998), Proc. 1998 Workshop on AI and Information Integration , AAAI Press.

[11]

Kushmerick, N. (2000), "Wrapper Induction: Efficiency and Expressiveness," J. Artificial Intelligence 118 , 1-2, 15-68.

Digital Library

[12]

Kushmerick, N., D. Weld, and R. Doorenbos (1997), "Wrapper Induction for Information Extraction," In Proc. 15th Int. Joint Conf. AI , pp. 729-35.

[13]

Levy, A., C. Knoblock, S. Minton, and W. Cohen (1998), "Trends and Controversies: Information Integration," IEEE Intelligent Systems 13 , 5, 12-24.

Digital Library

[14]

Muslea, I., S. Minton, and C. Knoblock (1998), "Wrapper Induction for Semi-structured, Web-based Information Sources," In Proc. Conf. Automatic Learning & Discovery .

[15]

Muslea, I., S. Minton, and C. Knoblock (1999), "A Hierachical Approach to Wrapper Induction," In Proc. 3rd Int. Conf. Autonomous Agents , pp. 190-197.

[16]

Rosenfeld, R. (1996), "A Maximum Entropy Approach to Adaptive Statistical Language Modelling," Computer, Speech and Language 10 , 3, 187-228.

[17]

Smith, D. and M. Lopez (1997), "Information Extraction for Semistructured Documents," In Proc. Workshop on Management of Semistructured Data .

[18]

Wiederhold, G. (1996), Intelligent Information Integration , Kluwer, Dordrecht.

Cited By

Burget R(2023)Scraping Data from Web Pages Using SPARQL QueriesWeb Engineering10.1007/978-3-031-34444-2_21(293-300)Online publication date: 6-Jun-2023
https://dl.acm.org/doi/10.1007/978-3-031-34444-2_21
Parameswaran ADalvi NGarcia-Molina HRastogi R(2020)Optimal schemes for robust web extractionProceedings of the VLDB Endowment10.14778/3402707.34027354:11(980-991)Online publication date: 3-Jun-2020
https://dl.acm.org/doi/10.14778/3402707.3402735
Ayala DBorrego AHernández IRuiz D(2020)A neural network for semantic labelling of structured informationExpert Systems with Applications: An International Journal10.1016/j.eswa.2019.113053143:COnline publication date: 1-Apr-2020
https://dl.acm.org/doi/10.1016/j.eswa.2019.113053
Show More Cited By

Index Terms

Wrapper verification

Index terms have been assigned to the content through auto-classification.

Recommendations

Wrapper maintenance: a machine learning approach

The proliferation of online information sources has led to an increased use of wrappers for extracting data from Web sources. While most of the previous research has focused on quick and efficient generation of wrappers, the development of tools for ...
Verification of attribute grammar
POPL '81: Proceedings of the 8th ACM SIGPLAN-SIGACT symposium on Principles of programming languages

Verification of attribute grammar is discussed. As is widely recognized, attribute grammar of Knuth [8] is a very convenient device to describe semantics of programming languages, especially in automating compiler construction. Many efforts have been ...
Applying ant colony hybrid metaheuristics to wrapper verification

First application of Ant Colony Metaheuristic to verify information extracted by web wrappers.New multilevel verification system that improves the results achieved by current techniques.Enumeration of current techniques weakness.Reformulation of wrapper ...

Comments

Information & Contributors

Information

Published In

cover image World Wide Web

World Wide Web Volume 3, Issue 2

Oct 2000

75 pages

ISSN:1386-145X

Issue’s Table of Contents

Copyright © Copyright © 2000 Kluwer Academic Publishers.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 15 March 2000

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

42
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Burget R(2023)Scraping Data from Web Pages Using SPARQL QueriesWeb Engineering10.1007/978-3-031-34444-2_21(293-300)Online publication date: 6-Jun-2023
https://dl.acm.org/doi/10.1007/978-3-031-34444-2_21
Parameswaran ADalvi NGarcia-Molina HRastogi R(2020)Optimal schemes for robust web extractionProceedings of the VLDB Endowment10.14778/3402707.34027354:11(980-991)Online publication date: 3-Jun-2020
https://dl.acm.org/doi/10.14778/3402707.3402735
Ayala DBorrego AHernández IRuiz D(2020)A neural network for semantic labelling of structured informationExpert Systems with Applications: An International Journal10.1016/j.eswa.2019.113053143:COnline publication date: 1-Apr-2020
https://dl.acm.org/doi/10.1016/j.eswa.2019.113053
Ayala DHernández IRuiz DToro M(2019)TAPON-MTInformation Systems10.1016/j.is.2018.12.00683:C(57-68)Online publication date: 1-Jul-2019
https://dl.acm.org/doi/10.1016/j.is.2018.12.006
Chang CChen TChen MDing J(2016)Efficient Page-Level Data Extraction via Schema Induction and VerificationProceedings, Part II, of the 20th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining - Volume 965210.1007/978-3-319-31750-2_38(478-490)Online publication date: 19-Apr-2016
https://dl.acm.org/doi/10.1007/978-3-319-31750-2_38
Sleiman HCorchuelo R(2014)A class of neural-network-based transducers for web information extractionNeurocomputing10.1016/j.neucom.2013.05.057135:C(61-68)Online publication date: 5-Jul-2014
https://dl.acm.org/doi/10.1016/j.neucom.2013.05.057
Thomsen JErnst EBrabrand CSchwartzbach M(2012)WebSelFProceedings of the 12th international conference on Web Engineering10.1007/978-3-642-31753-8_28(347-361)Online publication date: 23-Jul-2012
https://dl.acm.org/doi/10.1007/978-3-642-31753-8_28
Fazzinga BFlesca STagarelli A(2011)Schema-based Web wrappingKnowledge and Information Systems10.5555/3225627.322571626:1(127-173)Online publication date: 1-Jan-2011
https://dl.acm.org/doi/10.5555/3225627.3225716
Ferrara EBaumgartner R(2011)Intelligent self-repairable web wrappersProceedings of the 12th international conference on Artificial intelligence around man and beyond10.5555/2041977.2042009(274-285)Online publication date: 15-Sep-2011
https://dl.acm.org/doi/10.5555/2041977.2042009
Burget RBurgetova I(2011)Automatic annotation of online articles based on visual feature classificationInternational Journal of Intelligent Information and Database Systems10.1504/IJIIDS.2011.0413225:4(338-360)Online publication date: 1-Jul-2011
https://dl.acm.org/doi/10.1504/IJIIDS.2011.041322
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents