research-article

DIADEM: domain-centric, intelligent, automated data extraction methodology

Authors:

Giovanni Grasso,

Andrey Kravchenko,

Christian Schallhart,

Andrew Sellers,

Cheng WangAuthors Info & Claims

WWW '12 Companion: Proceedings of the 21st International Conference on World Wide Web

Pages 267 - 270

https://doi.org/10.1145/2187980.2188025

Published: 16 April 2012 Publication History

Abstract

Search engines are the sinews of the web. These sinews have become strained, however: Where the web's function once was a mix of library and yellow pages, it has become the central marketplace for information of almost any kind. We search more and more for objects with specific characteristics, a car with a certain mileage, an affordable apartment close to a good school, or the latest accessory for our phones. Search engines all too often fail to provide reasonable answers, making us sift through dozens of websites with thousands of offers--never to be sure a better offer isn't just around the corner. What search engines are missing is understanding of the objects and their attributes published on websites.

Automatically identifying and extracting these objects is akin to alchemy: transforming unstructured web information into highly structured data with near perfect accuracy. With DIADEM we present a formula for this transformation, but at a price: DIADEM identifies and extracts data from a website with high accuracy. The price is that for this task we need to provide DIADEM with extensive knowledge about the ontology and phenomenology of the domain, i.e., about entities (and relations) and about the representation of these entities in the textual, structural, and visual language of a website of this domain. In this demonstration, we demonstrate with a first prototype of DIADEM that, in contrast to alchemists, DIADEM has developed a viable formula.

References

[1]

M. Benedikt, G. Gottlob, and P. Senellart. Determining relevance of accesses at runtime. In PODS, 2011.

Digital Library

[2]

A. Calı, G. Gottlob, and A. Pieris. New expressive languages for ontological query answering. In AAAI, 2011.

[3]

A. Calı, G. Gottlob, and A. Pieris. Querying conceptual schemata with expressive equality constraints. In ER, 2011.

[4]

A. Calì, G. Gottlob, and A. Pieris. Query answering under non-guarded rules in datalog$^\pm$. In RR, 2010.

[5]

V. Crescenzi and G. Mecca. Automatic information extraction from large websites. J. ACM, 51(5), 2004.

Digital Library

[6]

N. N. Dalvi, R. Kumar, and M. A. Soliman. Automatic wrappers for large scale web extraction. In VLDB, 2011.

Digital Library

[7]

O. de Moor, G. Gottlob, T. Furche, and A. Sellers, eds. Datalog Reloaded, Revised Selected Papers, LNCS. 2011.

Digital Library

[8]

E. C. Dragut, T. Kabisch, C. Yu, and U. Leser. A hierarchical approach to model web query interfaces for web source integration. In VLDB, 2009.

Digital Library

[9]

T. Furche, G. Gottlob, G. Grasso, X. Guo, G. Orsi, and C. Schallhart. Real understanding of real estate forms. In WIMS, 2011.

Digital Library

[10]

T. Furche, G. Gottlob, G. Grasso, X. Guo, G. Orsi, and C. Schallhart. OPAL: Automated Form Understanding for the Deep Web. In WWW, 2012.

Digital Library

[11]

T. Furche, G. Gottlob, G. Grasso, G. Orsi, C. Schallhart, and C. Wang. Little knowledge rules the web: Domain-centric result page extraction. In RR, 2011.

Digital Library

[12]

T. Furche, G. Gottlob, X. Guo, C. Schallhart, A. Sellers, and C. Wang. How the minotaur turned into Ariadne: Ontologies in web data extraction. ICWE, (keynote), 2011.

Digital Library

[13]

T. Furche, G. Gottlob, G. Grasso, C. Schallhart, and A. Sellers. OXPath: A language for scalable, memory-efficient data extraction from web applications. In VLDB, 2011.

[14]

G. Gottlob, T. Lukasiewicz, and G. I. Simari. Answering threshold queries in probabilistic datalog - ontologies. In SUM, 2011.

Digital Library

[15]

G. Gottlob, T. Lukasiewicz, and G. I. Simari. Conjunctive query answering in probabilistic datalog - ontologies. In RR, 2011.

Digital Library

[16]

G. Gottlob, G. Orsi, and A. Pieris. Ontological queries: Rewriting and optimization. In ICDE, 2011.

Digital Library

[17]

N. Kushmerick. Wrapper induction: efficiency and expressiveness. AI, 118, 2000.

Digital Library

[18]

G. Orsi and A. Pieris. Optimizing query answering under ontological constraints. In VLDB, 2011.

[19]

A. Yates, M. Cafarella, M. Banko, O. Etzioni, M. Broadhead, and S. Soderland. Textrunner: open information extraction on the web. In NAACL 2007.

Digital Library

Cited By

Kuo LChang TLai C(2021)Application of visual colors in dynamic web page design through affective cognitionMultimedia Tools and Applications10.1007/s11042-021-11732-zOnline publication date: 8-Dec-2021
https://doi.org/10.1007/s11042-021-11732-z
Dong XHe XKan ALi XLiang YMa JXu YZhang CZhao TBlanco Saldana GDeshpande SMichetti Manduca ARen JSingh SXiao FChang HKaramanolakis GMao YWang YFaloutsos CMcCallum AHan JGupta RLiu YTang JPrakash B(2020)AutoKnowProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining10.1145/3394486.3403323(2724-2734)Online publication date: 20-Aug-2020
https://doi.org/10.1145/3394486.3403323
Dong X(2019)Building a Broad Knowledge Graph for Products2019 IEEE 35th International Conference on Data Engineering (ICDE)10.1109/ICDE.2019.00010(25-25)Online publication date: Apr-2019
https://doi.org/10.1109/ICDE.2019.00010
Show More Cited By

Index Terms

DIADEM: domain-centric, intelligent, automated data extraction methodology
1. Information systems
  1. World Wide Web
    1. Web applications
    2. Web services

Recommendations

DIADEM: thousands of websites to a single database

The web is overflowing with implicitly structured data, spread over hundreds of thousands of sites, hidden deep behind search forms, or siloed in marketplaces, only accessible as HTML. Automatic extraction of structured data at the scale of thousands of ...
Towards complete coverage in focused web harvesting
iiWAS '15: Proceedings of the 17th International Conference on Information Integration and Web-based Applications & Services

With the goal of harvesting all information about a given entity, in this paper, we try to harvest all matching documents for a given query submitted on a search engine. The objective is to retrieve all information about for instance "Michael Jackson", "...
Current challenges in web crawling
ICWE'13: Proceedings of the 13th international conference on Web Engineering

Web crawling, a process of collecting web pages in an automated manner, is the primary and ubiquitous operation used by a large number of web systems and agents starting from a simple program for website backup to a major web search engine. Due to an ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

WWW '12 Companion: Proceedings of the 21st International Conference on World Wide Web

April 2012

1250 pages

ISBN:9781450312301

DOI:10.1145/2187980

General Chairs:
Alain Mille
Université de Lyon, France
,
Fabien Gandon
INRIA, France
,
Jacques Misselis
HP, France
,
Program Chairs:
Michael Rabinovich
Case Western Reserve University, USA
,
Steffen Staab
University of Koblenz-Landau, Germany

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Univ. de Lyon: Universite de Lyon

In-Cooperation

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 April 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

WWW 2012

Sponsor:

Univ. de Lyon

WWW 2012: 21st World Wide Web Conference 2012

April 16 - 20, 2012

Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

30
Total Citations
View Citations
436
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Reflects downloads up to 29 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kuo LChang TLai C(2021)Application of visual colors in dynamic web page design through affective cognitionMultimedia Tools and Applications10.1007/s11042-021-11732-zOnline publication date: 8-Dec-2021
https://doi.org/10.1007/s11042-021-11732-z
Dong XHe XKan ALi XLiang YMa JXu YZhang CZhao TBlanco Saldana GDeshpande SMichetti Manduca ARen JSingh SXiao FChang HKaramanolakis GMao YWang YFaloutsos CMcCallum AHan JGupta RLiu YTang JPrakash B(2020)AutoKnowProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining10.1145/3394486.3403323(2724-2734)Online publication date: 20-Aug-2020
https://doi.org/10.1145/3394486.3403323
Dong X(2019)Building a Broad Knowledge Graph for Products2019 IEEE 35th International Conference on Data Engineering (ICDE)10.1109/ICDE.2019.00010(25-25)Online publication date: Apr-2019
https://doi.org/10.1109/ICDE.2019.00010
Kravchenko A(2019)Large-scale holistic approach to Web block classificationWorld Wide Web10.1007/s11280-018-0634-622:5(1999-2015)Online publication date: 1-Sep-2019
https://dl.acm.org/doi/10.1007/s11280-018-0634-6
Fayzrakhmanov RSallinger ESpencer BFurche TGottlob GChampin PGandon FMédini LLalmas MIpeirotis P(2018)Browserless Web Data ExtractionProceedings of the 2018 World Wide Web Conference10.1145/3178876.3186008(1095-1104)Online publication date: 10-Apr-2018
https://dl.acm.org/doi/10.1145/3178876.3186008
Yatskov AVarlamov MTurdakov D(2018)Extraction of Data from Mass Media Web SitesProgramming and Computing Software10.1134/S036176881805009244:5(344-352)Online publication date: 1-Sep-2018
https://dl.acm.org/doi/10.1134/S0361768818050092
Falkner AFriedrich GSchekotihin KTaupe RTeppan E(2018)Industrial Applications of Answer Set ProgrammingKI - Künstliche Intelligenz10.1007/s13218-018-0548-632:2-3(165-176)Online publication date: 13-Jun-2018
https://doi.org/10.1007/s13218-018-0548-6
Kravchenko A(2018)$${{\textsc {ber}}}_{y}{\textsc {l}}$$BERyL: A System for Web Block ClassificationTransactions on Computational Science XXXIII10.1007/978-3-662-58039-4_4(61-78)Online publication date: 16-Sep-2018
https://doi.org/10.1007/978-3-662-58039-4_4
Novella THolubová I(2018)User-Friendly and Extensible Web Data ExtractionAdvances in Information Systems Development10.1007/978-3-319-74817-7_14(225-241)Online publication date: 28-Mar-2018
https://doi.org/10.1007/978-3-319-74817-7_14
Zheleznyakov DKharlamov EHorrocks ISingh SMarkovitch S(2017)Trust-sensitive evolution of DL-Lite knowledge basesProceedings of the Thirty-First AAAI Conference on Artificial Intelligence10.5555/3298239.3298424(1266-1272)Online publication date: 4-Feb-2017
https://dl.acm.org/doi/10.5555/3298239.3298424
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten