tutorial

Datalog and emerging applications: an interactive tutorial

Authors:

Shan Shan Huang,

Todd Jeffrey Green,

Boon Thau LooAuthors Info & Claims

SIGMOD '11: Proceedings of the 2011 ACM SIGMOD International Conference on Management of data

Pages 1213 - 1216

https://doi.org/10.1145/1989323.1989456

Published: 12 June 2011 Publication History

Abstract

We are witnessing an exciting revival of interest in recursive Datalog queries in a variety of emerging application domains such as data integration, information extraction, networking, program analysis, security, and cloud computing. This tutorial briefly reviews the Datalog language and recursive query processing and optimization techniques, then discusses applications of Datalog in three application domains: data integration, declarative networking, and program analysis. Throughout the tutorial, we use LogicBlox, a commercial Datalog engine for enterprise software systems, to allow the audience to walk through code examples presented in the tutorial.

Supplementary Material

JPG File (tuesday14-athensview-tutorial2.jpg)

Download
12.15 KB

WMV File (tuesday14-athensview-tutorial2.wmv)

Download
433.25 MB

References

[1]

Amazon High Performance Computing Clusters. http://aws.amazon.com/ec2/hpc-applications. Accessed Dec 2010.

[2]

Intel Atom E600C with FPGA. http://www.intel.com/design/intarch/atom/index.htm. Accessed Dec 2010.

[3]

LogicBlox. http://www.logicblox.com. Accessed Dec 2010.

[4]

Semmle. http://www.semmle.com. Accessed Dec 2010.

[5]

Abiteboul, S., Abrams, Z., Haar, S., and Milo, T. Diagnosis of Asynchronous Discrete Event Systems-Datalog to the Rescue! In PODS (2005).

Digital Library

[6]

Abiteboul, S., Hull, R., and Vianu, V. Foundations of Databases. Addison-Wesley, 1995.

Digital Library

[7]

Alvaro, P., Condie, T., Conway, N., Elmeleegy, K., Hellerstein, J. M., and Sears, R. Boom analytics: exploring data-centric, declarative programming for the cloud. In EuroSys (2010).

Digital Library

[8]

Alvaro, P., Marczak, W., Conway, N., Hellerstein, J. M., Maier, D., and Sears, R. C. Dedalus: Datalog in time and space. Tech. Rep. UCB/EECS-2009--173, EECS Department, University of California, Berkeley, Dec 2009.

[9]

Arni, F., Ong, K., Tsur, S., Wang, H., and Zaniolo, C. The deductive database system LDL. TPLP 3, 1 (2003), 61--94.

Digital Library

[10]

Bancilhon, F., and Ramakrishnan, R. An amateur's introduction to recursive query processing strategies. SIGMOD Rec. 15, 2 (1986), 16--52.

Digital Library

[11]

Beeri, C., and Vardi, M. Y. A proof procedure for data dependencies. J. ACM 31, 4 (1984), 718--741.

Digital Library

[12]

Bidoit, N. Bases de Données Déductives: Présentation de Datalog. Armand Colin, 1992.

[13]

Bravenboer, M., and Smaragdakis, Y. Strictly declarative specification of sophisticated points-to analyses. In OOPSLA (2009).

Digital Library

[14]

Ceri, S., Gottlob, G., and Tanca, L. What you always wanted to know about datalog (and never dared to ask). TKDE 1, 1 (1989), 146--166.

Digital Library

[15]

de Moor, O., Sereni, D., Avgustinov, P., and Verbaere, M. Type inference for datalog and its application to query optimisation. In PODS (2008).

Digital Library

[16]

Deutsch, A., and Tannen, V. Reformulation of xml queries and constraints. In ICDT (2003), pp. 225--241.

Digital Library

[17]

Fagin, R., Kolaitis, P. G., Miller, R. J., and Popa, L. Data exchange: semantics and query answering. TCS 336, 1 (2005), 89--124.

Digital Library

[18]

Genesereth, M. R. Data Integration: The Relational Logic Approach. Morgan & Claypool Publishers, 2010.

Digital Library

[19]

Gottlob, G., Koch, C., Baumgartner, R., Herzog, M., and Flesca, S. The Lixto data extraction project: back and forth between theory and practice. In PODS (2004).

Digital Library

[20]

Govindaraju, N., Gray, J., Kumar, R., and Manocha, D. GPUTeraSort: high performance graphics co-processor sorting for large database management. In SIGMOD (2006).

Digital Library

[21]

Green, T. J., Huang, S. S., and Loo, B. T. Datalog and recursive query processing. Foundations and Trends in Databases (2011). In preparation.

[22]

Green, T. J., Karvounarakis, G., Ives, Z. G., and Tannen, V. Update exchange with mappings and provenance. In VLDB (2007).

Digital Library

[23]

Hajiyev, E., Verbaere, M., and de Moor, O. Codequest: Scalable source code queries with datalog. In ECOOP (2006).

Digital Library

[24]

He, B., Lu, M., Yang, K., Fang, R., Govindaraju, N. K., Luo, Q., and Sander, P. V. Relational query coprocessing on graphics processors. ACM TODS 34 (December 2009).

Digital Library

[25]

Jim, T. SD3: A Trust Management System With Certified Evaluation. In IEEE Symposium on Security and Privacy (May 2001).

Digital Library

[26]

Lenzerini, M. Data integration: A theoretical perspective. In PODS (2002), pp. 233--246.

Digital Library

[27]

Loo, B. T., Condie, T., Garofalakis, M., Gay, D. E., Hellerstein, J. M., Maniatis, P., Ramakrishnan, R., Roscoe, T., and Stoica, I. Declarative networking: Language, execution and optimization. In SIGMOD (2006).

Digital Library

[28]

Loo, B. T., Condie, T., Hellerstein, J. M., Maniatis, P., Roscoe, T., and Stoica, I. Implementing declarative overlays. In SOSP (2005).

Digital Library

[29]

Loo, B. T., Hellerstein, J. M., Stoica, I., and Ramakrishnan, R. Declarative routing: extensible routing with declarative queries. In SIGCOMM (2005).

Digital Library

[30]

Ludascher, B., May, W., and Lausen, G. Nested transactions in a logical language for active rules. In LID (1996).

Digital Library

[31]

Marczak, W. R., Huang, S. S., Bravenboer, M., Sherr, M., Loo, B. T., and Aref, M. Secureblox: customizable secure distributed data processing. In SIGMOD (2010).

Digital Library

[32]

Mengmeng Liu and Nicholas Taylor and Wenchao Zhou and Zachary Ives and Boon Thau Loo. Recursive Computation of Regions and Connectivity in Networks. In ICDE (2009).

Digital Library

[33]

Mueller, R., Teubner, J., and Alonso, G. Data processing on FPGAs. PVLDB 2, 1 (August 2009).

Digital Library

[34]

Mueller, R., Teubner, J., and Alonso, G. Glacier: a query-to-hardware compiler. In SIGMOD (2010).

Digital Library

[35]

Ramakrishnan, R., Srivastava, D., Sudarshan, S., and Seshadri, P. The CORAL deductive system. The VLDB Journal 3, 2 (1994), 161--210.

Digital Library

[36]

Ramakrishnan, R., and Ullman, J. D. A Survey of Research on Deductive Database Systems. JLP 23, 2 (1993).

[37]

Ross, K. A syntactic stratification condition using constraints. In ILPS (1994).

Digital Library

[38]

Shen, W., Doan, A., Naughton, J., and Ramakrishnan, R. Declarative information extraction using datalog with embedded extraction predicates. In VLDB (2007).

Digital Library

[39]

Stonebraker, M., and Hellerstein, J. M., Eds. Readings in Database Systems, Third Edition. Morgan Kaufmann, 1998.

Digital Library

[40]

Ullman, J. D. Principles of Database and Knowledge-Base Systems, vol. II. W. H. Freeman & Co., 1990.

Digital Library

Cited By

Fejza AGenevès PLayaïda N(2024)Efficient Enumeration of Recursive Plans in Transformation-Based Query OptimizersProceedings of the VLDB Endowment10.14778/3681954.368198617:11(3095-3108)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.14778/3681954.3681986
Klopp DErdweg SPacak A(2024)Object-Oriented Fixpoint Programming with DatalogProceedings of the ACM on Programming Languages10.1145/36897138:OOPSLA2(60-86)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3689713
St. Amour LTilevich EErtl MKirsch C(2024)Toward Declarative Auditing of Java Software for Graceful Exception HandlingProceedings of the 21st ACM SIGPLAN International Conference on Managed Programming Languages and Runtimes10.1145/3679007.3685057(90-97)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3679007.3685057
Show More Cited By

Index Terms

Datalog and emerging applications: an interactive tutorial
1. Information systems
  1. Data management systems
    1. Query languages
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Database theory
      1. Database query languages (principles)

Recommendations

Query containment for data integration systems
Special issue on PODS 2000

The problem of query containment is fundamental to many aspects of database systems, including query optimization, determining independence of queries from updates, and rewriting queries using views. In the data-integration framework, however, the ...
Consistent Query Answering for Primary Keys in Datalog
Abstract
We study the complexity of consistent query answering on databases that may violate primary key constraints. A repair of such a database is any consistent database that can be obtained by deleting a minimal set of tuples. For every Boolean query q,...
The Complexity of Why-Provenance for Datalog Queries
PODS

Datalog is a powerful rule-based language that allows us to express complex recursive queries and has found numerous applications over the years. Explaining why a result to a Datalog query is obtained is an essential task towards explainable and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGMOD '11: Proceedings of the 2011 ACM SIGMOD International Conference on Management of data

June 2011

1364 pages

ISBN:9781450306614

DOI:10.1145/1989323

General Chair:
Timos Sellis
IMIS/RC Athena
,
Program Chair:
Renée J. Miller
University of Toronto
,
Publications Chairs:
Anastasios Kementsietsidis
IBM T.J. Watson Research Center
,
Yannis Velegrakis
University of Trento

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOD: ACM Special Interest Group on Management of Data

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 June 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Tutorial

Conference

SIGMOD/PODS '11

Sponsor:

SIGMOD

SIGMOD/PODS '11: International Conference on Management of Data

June 12 - 16, 2011

Athens, Greece

Acceptance Rates

Overall Acceptance Rate 785 of 4,003 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

94
Total Citations
View Citations
927
Total Downloads

Downloads (Last 12 months)47
Downloads (Last 6 weeks)6

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Fejza AGenevès PLayaïda N(2024)Efficient Enumeration of Recursive Plans in Transformation-Based Query OptimizersProceedings of the VLDB Endowment10.14778/3681954.368198617:11(3095-3108)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.14778/3681954.3681986
Klopp DErdweg SPacak A(2024)Object-Oriented Fixpoint Programming with DatalogProceedings of the ACM on Programming Languages10.1145/36897138:OOPSLA2(60-86)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3689713
St. Amour LTilevich EErtl MKirsch C(2024)Toward Declarative Auditing of Java Software for Graceful Exception HandlingProceedings of the 21st ACM SIGPLAN International Conference on Managed Programming Languages and Runtimes10.1145/3679007.3685057(90-97)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3679007.3685057
Dorn CPribadi H(2024)TypeQL: A Type-Theoretic & Polymorphic Query LanguageProceedings of the ACM on Management of Data10.1145/36516112:2(1-27)Online publication date: 14-May-2024
https://dl.acm.org/doi/10.1145/3651611
Shaikhha ASuciu DSchleich MNgo H(2024)Optimizing Nested Recursive QueriesProceedings of the ACM on Management of Data10.1145/36392712:1(1-27)Online publication date: 26-Mar-2024
https://dl.acm.org/doi/10.1145/3639271
Wang RChen KZhang CPan ZLi QQin SXu SZhang MLi YCalandrino JTroncoso C(2023)AlphaEXPProceedings of the 32nd USENIX Conference on Security Symposium10.5555/3620237.3620474(4229-4246)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.5555/3620237.3620474
Duong HHo VDo P(2023)Fact-checking Vietnamese Information Using Knowledge Graph, Datalog, and KG-BERTACM Transactions on Asian and Low-Resource Language Information Processing10.1145/362455722:10(1-23)Online publication date: 13-Oct-2023
https://dl.acm.org/doi/10.1145/3624557
Duval DEchahed RProst F(2023)A Rule-Based Procedure for Graph Query SolvingGraph Transformation10.1007/978-3-031-36709-0_9(163-183)Online publication date: 12-Jul-2023
https://doi.org/10.1007/978-3-031-36709-0_9
Zhao DSubotić PRaghothaman MScholz B(2023)Automatic Rollback Suggestions for Incremental Datalog EvaluationPractical Aspects of Declarative Languages10.1007/978-3-031-24841-2_19(295-312)Online publication date: 8-Jan-2023
https://doi.org/10.1007/978-3-031-24841-2_19
Pacak ASzabó TErdweg SScholz BKameyama Y(2022)Incremental Processing of Structured Data in DatalogProceedings of the 21st ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences10.1145/3564719.3568686(20-32)Online publication date: 29-Nov-2022
https://dl.acm.org/doi/10.1145/3564719.3568686
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents