Article

Webpage understanding: an integrated approach

Authors:

Hsiao-Wuen HonAuthors Info & Claims

KDD '07: Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining

Pages 903 - 912

https://doi.org/10.1145/1281192.1281288

Published: 12 August 2007 Publication History

Abstract

Recent work has shown the effectiveness of leveraging layout and tag-tree structure for segmenting webpages and labeling HTML elements. However, how to effectively segment and label the text contents inside HTML elements is still an open problem. Since many text contents on a webpage are often text fragments and not strictly grammatical, traditional natural language processing techniques, that typically expect grammatical sentences, are no longer directly applicable. In this paper, we examine how to use layout and tag-tree structure in a principled way to help understand text contents on webpages. We propose to segment and label the page structure and the text content of a webpage in a joint discriminative probabilistic model. In this model, semantic labels of page structure can be leveraged to help text content understanding, and semantic labels ofthe text phrases can be used in page structure understanding tasks such as data record detection. Thus, integration of both page structure and text content understanding leads to an integrated solution of webpage understanding. Experimental results on research homepage extraction show the feasibility and promise of our approach.

Supplementary Material

JPG File (p903-zhu-200.jpg)

Download
6.25 KB

JPG File (p903-zhu-768.jpg)

Download
8.19 KB

Low Resolution (p903-zhu-200.mov)

Download
29.43 MB

High Resolution (p903-zhu-768.mov)

Download
99.11 MB

References

[1]

A. Arasu and H. Garcia-Molina. Extracting Structured Data from Webpages. Proc. of SIGMOD, 2003.

Digital Library

[2]

V. Borkar, K. Deshmukh and S. Sarawagi. Automatic segmentation of text into structured records. Proc. of SIGMOD, 2001.

Digital Library

[3]

M. E. Califf and R. J. Mooney. Bottom-up relational learning of pattern matching rules for information extraction. Journal of Machine Learning Research, 2004.

Digital Library

[4]

C.-H. Chang and S.-L. Liu. IEPAD: Information Extraction Based on Pattern Discovery. Proc. of WWW, 2001.

Digital Library

[5]

W. W. Cohen and S. Sarawagi. Exploiting Dictionaries in Named Entity Extraction: Combining Semi-Markov Extraction Processes and Data Integration Methods. Proc. of SIGKDD, 2004.

Digital Library

[6]

V. Crescenzi, G. Mecca and P. Merialdo. ROADRUNNER: Towards Automatic Data Extraction from Large Web Sites. Proc. of VLDB, 2001.

Digital Library

[7]

B. Crysmann, A. Frank, B. Kiefer, S. Muller, G. Neumann, J. Piskorski, U. Schafer, M. Siegel, H. Uszkoreit, F. Xu, M. Becher and H-U. Krieger. An Integrated Architecture for Shallow and Deep Processing. Proc. of ACL, 2004.

Digital Library

[8]

D. W. Embley, Y. Jiang and Y.-K. Ng. Record-Boundary Discovery in Web Documents. Proc. of SIGMOD, 1999.

Digital Library

[9]

D. DiPasquo. Using HTML Formatting to Aid in Natural Language Processing on the World Wide Web. Senior Honors Thesis, Carnegie Mellon University, 1998.

[10]

T. Duong, H. Bui, D. Phung and S. Venkatesh. Activity Recognition and Abnormality Detection with the Switching Hidden Semi-Markov Model. Proc. of CVPR, 2005.

Digital Library

[11]

D. Freitag. Information Extraction from HTML: Application of a General Machine Learning Approach. Proc. of AAAI, 1998.

Digital Library

[12]

C. Jacquemin and C. Bush. Combining Lexical and Formatting Cues for Named Entity Acquisition from the Web. Proc. of the Joint SIGDAT conference on Empirical methods in natural language processing and very large corpora, 2000.

Digital Library

[13]

F. V. Jensen, S. L. Lauritzen and K. G. Olesen. Bayesian updating in causal probabilistic networks by local computation. Computational Statistics Quarterly, 4:269--82, 1990.

[14]

N. Kushmerick. Wrapper induction: efficiency and expressiveness. Artificial Intelligence, 118:15--68, 2000.

Digital Library

[15]

J. Lafferty, A. McCallum and F. Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. Proc. of ICML, 2001.

Digital Library

[16]

K. Lerman, L. Getoor, S. Minton and C. Knoblock. Using the Structure of Web Sites for Automatic Segmentation of Tables. Proc. of SIGMOD, 2004.

Digital Library

[17]

K. Lerman, S. Minton and C. Knoblock. Wrapper maintenance: A machine learning approach. Journal of Artificial Intelligence Research, 18:149--181, 2003.

Digital Library

[18]

C. Manning and H. Schutze. Foundations of Statistical Natural Language Processing. The MIT Press Cambridge, MA, May, 1999.

Digital Library

[19]

D. Phung, T. Duong, S. Venkatesh and H. Bui. Topic Transition Detection Using Hierarchical Hidden Markov and Semi-Markov Models. Proc. of MM, 2005.

Digital Library

[20]

S. Sarawagi and W. W. Cohen. Semi-Markov Conditional Random Fields for Information Extraction. Proc. of NIPS, 2004.

[21]

S. Sarawagi. Efficient Inference on Sequence Segmentation Models. Proc. of ICML, 2006.

Digital Library

[22]

S. Soderland. Learning to Extract Text-based Information from the World Wide Web. Proc. of SIGKDD, 1997.

[23]

S. Soderland. Learning Information Extraction Rules for Semi-structured and Free Text. Journal of Machine Learning, 1999.

Digital Library

[24]

F. Suchanek, G. Ifrim and G. Weikum. Combining Linguistic and Statistical Analysis to Extract Relations from Web Documents. Proc. of SIGKDD, 2006.

Digital Library

[25]

T. Yoshimasa and T. Jun'ichi. Chunk Parsing Revisited. Proc. of the 9th International Workshop on Parsing Technologies, 2005.

Digital Library

[26]

Y. Zhai and B. Liu. Web Data Extraction Based on Partial Tree Alignment. Proc. of WWW, 2005.

Digital Library

[27]

H. Zhao, W. Meng, Z. Wu, V. Raghavan and C. Yu. Fully Automatic Wrapper Generation for Search Engines. Proc. of WWW, 2005.

Digital Library

[28]

J. Zhu, Z. Nie, J.-R. Wen, B. Zhang and W.-Y. Ma. 2D Conditional Random Fields for Web Information Extraction. Proc. of ICML, 2005.

Digital Library

[29]

J. Zhu, Z. Nie, J.-R. Wen, B. Zhang and W.-Y. Ma. Simultaneous Record Detection and Attribute Labeling in Web Data Extraction. Proc. of SIGKDD, 2006.

Digital Library

Cited By

Yuan ZChen BDai YLi YZheng HZhang R(2024)An Anchor Learning Approach for Citation Field LearningICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10448007(12346-12350)Online publication date: 14-Apr-2024
https://doi.org/10.1109/ICASSP48485.2024.10448007
Zhang ZYu BLiu TLiu TWang YGuo L(2023)Learning Structural Co-occurrences for Structured Web Data Extraction in Low-Resource SettingsProceedings of the ACM Web Conference 202310.1145/3543507.3583387(1683-1692)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543507.3583387
Xie KWang CWang P(2021)A Domain-Independent Ontology Learning Method Based on Transfer LearningElectronics10.3390/electronics1016191110:16(1911)Online publication date: 9-Aug-2021
https://doi.org/10.3390/electronics10161911
Show More Cited By

Recommendations

Webpage understanding: beyond page-level search

In this paper we introduce the webpage understanding problem which consists of three subtasks: webpage segmentation, webpage structure labeling, and webpage text segmentation and labeling. The problem is motivated by the search applications we have been ...
Closing the Loop in Webpage Understanding

The two most important tasks in information extraction from the Web are webpage structure understanding and natural language sentences processing. However, little work has been done toward an integrated statistical model for understanding webpage ...
Closing the loop in webpage understanding
CIKM '08: Proceedings of the 17th ACM conference on Information and knowledge management

Little work has been done towards an integrated statistical model for understanding webpage structures and processing natural language sentences within the HTML elements. This paper proposed a novel framework called WebNLP which enables bidirectional ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '07: Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining

August 2007

1080 pages

ISBN:9781595936097

DOI:10.1145/1281192

General Chair:
Pavel Berkhin
Yahoo!, USA
,
Program Chairs:
Rich Caruana
Cornell University, USA
,
Xindong Wu
University of Vermont, USA

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 August 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

KDD07

Sponsor:

KDD07: The 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

August 12 - 15, 2007

California, San Jose, USA

Acceptance Rates

KDD '07 Paper Acceptance Rate 111 of 573 submissions, 19%;

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
1,090
Total Downloads

Downloads (Last 12 months)18
Downloads (Last 6 weeks)1

Reflects downloads up to 15 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yuan ZChen BDai YLi YZheng HZhang R(2024)An Anchor Learning Approach for Citation Field LearningICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10448007(12346-12350)Online publication date: 14-Apr-2024
https://doi.org/10.1109/ICASSP48485.2024.10448007
Zhang ZYu BLiu TLiu TWang YGuo L(2023)Learning Structural Co-occurrences for Structured Web Data Extraction in Low-Resource SettingsProceedings of the ACM Web Conference 202310.1145/3543507.3583387(1683-1692)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543507.3583387
Xie KWang CWang P(2021)A Domain-Independent Ontology Learning Method Based on Transfer LearningElectronics10.3390/electronics1016191110:16(1911)Online publication date: 9-Aug-2021
https://doi.org/10.3390/electronics10161911
Zhang YDai YQi JXu XZhang R(2018)Citation Field Learning by RNN with Limited Training DataTrends and Applications in Knowledge Discovery and Data Mining10.1007/978-3-030-04503-6_23(219-232)Online publication date: 21-Nov-2018
https://doi.org/10.1007/978-3-030-04503-6_23
Mo WWang PSong HZhao JZhang X(2013)Learning Domain-Specific Ontologies from the WebLinked Data and Knowledge Graph10.1007/978-3-642-54025-7_12(132-146)Online publication date: 2013
https://doi.org/10.1007/978-3-642-54025-7_12
Wang JLiu ZZhao H(2012)A probabilistic model with multi-dimensional features for object extractionFrontiers of Computer Science10.1007/s11704-012-1093-36:5(513-526)Online publication date: 10-Oct-2012
https://doi.org/10.1007/s11704-012-1093-3
Yu XKing ILyu M(2011)Towards a top-down and bottom-up bidirectional approach to joint information extractionProceedings of the 20th ACM international conference on Information and knowledge management10.1145/2063576.2063699(847-856)Online publication date: 24-Oct-2011
https://dl.acm.org/doi/10.1145/2063576.2063699
Kulkarni CKlemmer STan DBegole BKellogg W(2011)Automatically adapting web pages to heterogeneous devicesCHI '11 Extended Abstracts on Human Factors in Computing Systems10.1145/1979742.1979810(1573-1578)Online publication date: 7-May-2011
https://dl.acm.org/doi/10.1145/1979742.1979810
Wong TBing LLam WKing INejdl WLi H(2011)Normalizing web product attributes and discovering domain ontology with minimal effortProceedings of the fourth ACM international conference on Web search and data mining10.1145/1935826.1935931(805-814)Online publication date: 9-Feb-2011
https://dl.acm.org/doi/10.1145/1935826.1935931
Yu XLam W(2011)Probabilistic joint models incorporating logic and learning via structured variational approximation for information extractionKnowledge and Information Systems10.1007/s10115-011-0455-832:2(415-444)Online publication date: 10-Nov-2011
https://doi.org/10.1007/s10115-011-0455-8
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents