Article

Mining data records in Web pages

Authors:

Bing Liu,

Robert Grossman,

Yanhong ZhaiAuthors Info & Claims

KDD '03: Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining

Pages 601 - 606

https://doi.org/10.1145/956750.956826

Published: 24 August 2003 Publication History

Get Access

Abstract

A large amount of information on the Web is contained in regularly structured objects, which we call data records. Such data records are important because they often present the essential information of their host pages, e.g., lists of products or services. It is useful to mine such data records in order to extract information from them to provide value-added services. Existing automatic techniques are not satisfactory because of their poor accuracies. In this paper, we propose a more effective technique to perform the task. The technique is based on two observations about data records on the Web and a string matching algorithm. The proposed technique is able to mine both contiguous and non-contiguous data records. Our experimental results show that the proposed technique outperforms existing techniques substantially.

References

[1]

Baeza-Yates, R. "Algorithms for string matching: A survey." ACM SIGIR Forum, 23(3--4):34--58, 1989

Digital Library

Google Scholar

[2]

Buttler, D., Liu, L., Pu, C. "A fully automated extraction system for the World Wide Web." IEEE ICDCS-21, 2001.

Google Scholar

[3]

Chang, C-H., Lui, S-L. "IEPAD: Information extraction based on pattern discovery." WWW-10, 2001.

Digital Library

Google Scholar

[4]

Cohen, W., Hurst, M., and Jensen, L. "A flexible learning system for wrapping tables and lists in HTML documents." WWW-2002, 2002.

Digital Library

Google Scholar

[5]

Doorenbos, R., Etzioni, O., Weld, D. "A scalable comparison shopping agent for the World Wide Web." Agents-97, 1997.

Digital Library

Google Scholar

[6]

Embley, D., Jiang, Y and Ng, Y. "Record-boundary discovery in Web documents," SIGMOD-99, 1999.

Digital Library

Google Scholar

[7]

Gusfield, D. Algorithms on strings, tree, and sequence. 1997.

Digital Library

Google Scholar

[8]

Hsu, C.-N., and Dung, M.-T. "Generating finite-state transducers for semi-structured data extraction from the Web." Information Systems. 23(8): 521--538, 1998.

Digital Library

Google Scholar

[9]

Kushmerick, N. "Wrapper induction: efficiency and expressiveness." Artificial Intelligence, 118:15--68, 2000.

Digital Library

Google Scholar

[10]

Lerman, K. Knoblock, C., and Minton, S. "Automatic data extraction from lists and tables in web sources." IJCAI-01 Workshop on Adaptive Text Extraction and Mining, 2001.

Google Scholar

[11]

Liu, B., Grossman, R. and Zhai, Y. "Mining data records in Web pages." UIC Technical Report, 2003.

Google Scholar

[12]

Muslea, I., Minton, S. and Knoblock, C. "A hierarchical approach to wrapper induction." Agents-99, 1999.

Digital Library

Google Scholar

Cited By

View all

Lin YChang CChuang HLin XYeh THong M(2024)Cost-Effective Event Mining on the Web via Event Source Page Discovery and Data API ConstructionIEEE Access10.1109/ACCESS.2024.344544812(115981-115993)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3445448
Nunes MDorneles C(2023)EDREW - Enhanced Data Representation for Extraction in WebProceedings of the 29th Brazilian Symposium on Multimedia and the Web10.1145/3617023.3617055(230-237)Online publication date: 23-Oct-2023
https://dl.acm.org/doi/10.1145/3617023.3617055
Chen ZMeng WDragut E(2022)Web Record Extraction with InvariantsProceedings of the VLDB Endowment10.14778/3574245.357427616:4(959-972)Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.14778/3574245.3574276
Show More Cited By

Index Terms

Mining data records in Web pages
1. Computing methodologies
  1. Machine learning
2. Information systems
  1. Information systems applications

Recommendations

Extraction of flat and nested data records from web pages
AusDM '06: Proceedings of the fifth Australasian conference on Data mining and analystics - Volume 61

This paper deals with studies the problem of identification and extraction of flat and nested data records from a given web page. With the explosive growth of information sources available on the World Wide Web, it has become increasingly difficult to ...
Mining Web Pages for Data Records

Much information on the Web is contained in regularly structured objects, or data records. Data records often present their host pages' essential information, such as lists of products and services. Mining data records to extract this information can ...
Web data mining: exploring hyperlinks, contents, and usage data

This paper presents a review of the book "Web Data Mining - Exploring Hyperlinks, Contents, and Usage Data" by Bing Liu. The review concludes that the breadth and depth of this book makes it a required staple for every Web mining researcher, student, or ...

Comments

Information & Contributors

Information

Published In

KDD '03: Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining

August 2003

736 pages

ISBN:1581137370

DOI:10.1145/956750

Conference Chair:
Lise Getoor
University of Maryland, College Park
,
General Chair:
Ted Senator
DARPA
,
Program Chairs:
Pedro Domingos
University of Washington
,
Christos Faloutsos
Carnegie Mellon University

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 August 2003

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

KDD03

Sponsor:

KDD03: The Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

August 24 - 27, 2003

Washington, D.C.

Acceptance Rates

KDD '03 Paper Acceptance Rate 46 of 298 submissions, 15%;

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

236
Total Citations
View Citations
2,442
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)4

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Lin YChang CChuang HLin XYeh THong M(2024)Cost-Effective Event Mining on the Web via Event Source Page Discovery and Data API ConstructionIEEE Access10.1109/ACCESS.2024.344544812(115981-115993)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3445448
Nunes MDorneles C(2023)EDREW - Enhanced Data Representation for Extraction in WebProceedings of the 29th Brazilian Symposium on Multimedia and the Web10.1145/3617023.3617055(230-237)Online publication date: 23-Oct-2023
https://dl.acm.org/doi/10.1145/3617023.3617055
Chen ZMeng WDragut E(2022)Web Record Extraction with InvariantsProceedings of the VLDB Endowment10.14778/3574245.357427616:4(959-972)Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.14778/3574245.3574276
Qi PSun YLuo HGuizani M(2022)Scratch-DKG: A Framework for Constructing Scratch Domain Knowledge GraphIEEE Transactions on Emerging Topics in Computing10.1109/TETC.2020.299671010:1(170-185)Online publication date: 1-Jan-2022
https://doi.org/10.1109/TETC.2020.2996710
Chang CLiao YYeh T(2022)Event Source Page Discovery via Policy-Based RL with Multi-task Neural Sequence ModelWeb Information Systems Engineering – WISE 202210.1007/978-3-031-20891-1_42(597-606)Online publication date: 7-Nov-2022
https://doi.org/10.1007/978-3-031-20891-1_42
Chang CWu CLin T(2022)Automatic Web Data API Creation via Cross-Lingual Neural Pagination RecognitionWeb Engineering10.1007/978-3-031-09917-5_8(117-131)Online publication date: 1-Jul-2022
https://doi.org/10.1007/978-3-031-09917-5_8
Patnaik SNarendra Babu C(2021)Trends in web data extraction using machine learningWeb Intelligence10.3233/WEB-210465(1-22)Online publication date: 15-Nov-2021
https://doi.org/10.3233/WEB-210465
Yang ZZhang MZhang ZLi HLiu CAli S(2021)Lecture Information Service Based on Multiple Features FusionInternational Journal of Software Engineering and Knowledge Engineering10.1142/S021819402140007631:04(545-562)Online publication date: 18-May-2021
https://doi.org/10.1142/S0218194021400076
Kumar SKumar R(2021)A Study on Different Aspects of Web Mining and Research IssuesIOP Conference Series: Materials Science and Engineering10.1088/1757-899X/1022/1/0120181022(012018)Online publication date: 19-Jan-2021
https://doi.org/10.1088/1757-899X/1022/1/012018
Faisal Noori SBazeer Ahamed B(2021)A framework enhancement method of deep web data extractionMaterials Today: Proceedings10.1016/j.matpr.2021.01.132Online publication date: Feb-2021
https://doi.org/10.1016/j.matpr.2021.01.132
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Extraction of flat and nested data records from web pages

Mining Web Pages for Data Records

Web data mining: exploring hyperlinks, contents, and usage data