Article

PADS: an end-to-end system for processing ad hoc data

Authors:

Xuan ZhengAuthors Info & Claims

SIGMOD '06: Proceedings of the 2006 ACM SIGMOD international conference on Management of data

Pages 727 - 729

https://doi.org/10.1145/1142473.1142568

Published: 27 June 2006 Publication History

Get Access

Abstract

Enormous amounts of data exist in "well-behaved" formats such as relational tables and XML, which come equipped with extensive tool support. However, vast amounts of data also exist in non-standard or ad hoc data formats, which often lack standard or extensible tools. This deficiency forces data analysts to implement their own tools for parsing, querying, and analyzing their ad hoc data. The resulting tools typically interleave parsing, querying, and analysis, obscuring the semantics of the data format and making it nearly impossible for others to resuse the tools. This proposal describes PADS, an end-to-end system for processing ad hoc data sources. The core of PADS is a declarative language for describing ad hoc data sources and a data-description compiler that produces customizable libraries for parsing the ad hoc data. A suite of tools built around this core includes statistical data-profiling tools, a query engine that permits viewing ad hoc sources as XML and for querying them with XQuery, and an interactive front-end that helps users produce PADS descriptions quickly.

References

[1]

Galax user manual. http://www.galaxquery.org.

Google Scholar

[2]

PADS user manual. http://www.padsproj.org/.

Google Scholar

[3]

C. Cranor, Y. Gao, T. Johnson, V. Shkapenyuk, and O. Spatscheck. Gigascope: High performance network monitoring with an SQL interface. In SIGMOD. ACM, 2002.

Digital Library

Google Scholar

[4]

M. Fernández, K. Fisher, R. Gruber, and Y. Mandelbaum. PADX: Querying large-scale ad hoc data with XQuery. In PLAN-X, 2006.

Google Scholar

[5]

K. Fisher and R. Gruber. PADS: A domain-specific language for processing ad hoc data. In PLDI, 2005.

Digital Library

Google Scholar

[6]

K. Fisher, Y. Mandelbaum, and D. Walker. The next 700 data description languages. In POPL, 2006.

Digital Library

Google Scholar

[7]

A. C. Gilbert, S. Guha, P. Indyk, Y. Kotidis, S. Muthukrishnan, and M. Strauss. Fast, small-space algorithms for approximate histogram maintenance. In STOC, 2002.

Digital Library

Google Scholar

[8]

A. C. Gilbert, Y. Kotidis, S. Muthukrishnan, and M. Strauss. How to summarize the universe: Dynamic maintenance of quantiles. In VLDB, 2002.

Digital Library

Google Scholar

[9]

S. Guha, P. Indyk, S. Muthukrishnan, and M. Strauss. Histogramming data streams with fast per-item processing. In ICALP, 2002.

Digital Library

Google Scholar

Cited By

View all

Drăgoi CWidder JZufferey D(2020)Programming at the edge of synchronyProceedings of the ACM on Programming Languages10.1145/34282814:OOPSLA(1-30)Online publication date: 13-Nov-2020
https://dl.acm.org/doi/10.1145/3428281
Gulwani SLe VRadhakrishna ARadiček IRaza M(2020)Structure interpretation of text formatsProceedings of the ACM on Programming Languages10.1145/34282804:OOPSLA(1-29)Online publication date: 13-Nov-2020
https://dl.acm.org/doi/10.1145/3428280
Newcomb JAdams AJohnson SBodik RKamil S(2020)Verifying and improving Halide’s term rewriting system with program synthesisProceedings of the ACM on Programming Languages10.1145/34282344:OOPSLA(1-28)Online publication date: 13-Nov-2020
https://dl.acm.org/doi/10.1145/3428234
Show More Cited By

Index Terms

PADS: an end-to-end system for processing ad hoc data
1. Information systems
  1. Data management systems
    1. Database design and models
    2. Database management system engines
      1. Database query processing
  2. Information retrieval
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Database theory
      1. Database query processing and optimization (theory)

Recommendations

PADS: a domain-specific language for processing ad hoc data
Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation

PADS is a declarative data description language that allows data analysts to describe both the physical layout of ad hoc data sources and semantic properties of that data. From such descriptions, the PADS compiler generates libraries and tools for ...
PADS: a domain-specific language for processing ad hoc data
PLDI '05: Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation

PADS is a declarative data description language that allows data analysts to describe both the physical layout of ad hoc data sources and semantic properties of that data. From such descriptions, the PADS compiler generates libraries and tools for ...
PADS/ML: a functional data description language
POPL '07: Proceedings of the 34th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages

Massive amounts of useful data are stored and processed in ad hoc formats for which common tools like parsers, printers, query engines and format converters are not readily available. In this paper, we explain the design and implementation of PADS/ML , a ...

Comments

Information & Contributors

Information

Published In

SIGMOD '06: Proceedings of the 2006 ACM SIGMOD international conference on Management of data

June 2006

830 pages

ISBN:1595934340

DOI:10.1145/1142473

General Chairs:
Clement Yu
University of Illinois at Chicago
,
Peter Scheuermann
Northwestern University
,
Program Chair:
Surajit Chaudhuri
Microsoft Research

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 June 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

SIGMOD/PODS06

Sponsor:

SIGMOD/PODS06: International Conference on Management of Data and Symposium on Principles Database and Systems

June 27 - 29, 2006

IL, Chicago, USA

Acceptance Rates

Overall Acceptance Rate 785 of 4,003 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
304
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)1

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Drăgoi CWidder JZufferey D(2020)Programming at the edge of synchronyProceedings of the ACM on Programming Languages10.1145/34282814:OOPSLA(1-30)Online publication date: 13-Nov-2020
https://dl.acm.org/doi/10.1145/3428281
Gulwani SLe VRadhakrishna ARadiček IRaza M(2020)Structure interpretation of text formatsProceedings of the ACM on Programming Languages10.1145/34282804:OOPSLA(1-29)Online publication date: 13-Nov-2020
https://dl.acm.org/doi/10.1145/3428280
Newcomb JAdams AJohnson SBodik RKamil S(2020)Verifying and improving Halide’s term rewriting system with program synthesisProceedings of the ACM on Programming Languages10.1145/34282344:OOPSLA(1-28)Online publication date: 13-Nov-2020
https://dl.acm.org/doi/10.1145/3428234
Zhang HRoth EHaeberlen APierce BRoth A(2020)Testing differential privacy with dual interpretersProceedings of the ACM on Programming Languages10.1145/34282334:OOPSLA(1-26)Online publication date: 13-Nov-2020
https://dl.acm.org/doi/10.1145/3428233
Bhat KKumar N(2020)Sociocultural Dimensions of Tracking Health and Taking CareProceedings of the ACM on Human-Computer Interaction10.1145/34152004:CSCW2(1-24)Online publication date: 15-Oct-2020
https://dl.acm.org/doi/10.1145/3415200
Ashktorab ZLiao QDugan CJohnson JPan QZhang WKumaravel SCampbell M(2020)Human-AI Collaboration in a Cooperative Game SettingProceedings of the ACM on Human-Computer Interaction10.1145/34151674:CSCW2(1-20)Online publication date: 15-Oct-2020
https://dl.acm.org/doi/10.1145/3415167
Mundkur PBriesemeister LShankar NAnantharaman PAli SLucas ZSmith S(2020)Research Report: The Parsley Data Format Definition Language2020 IEEE Security and Privacy Workshops (SPW)10.1109/SPW50608.2020.00064(300-307)Online publication date: May-2020
https://doi.org/10.1109/SPW50608.2020.00064
Reiher TSenier ACastrillon JStrufe T(2020)RecordFlux: Formal Message Specification and Generation of Verifiable Binary ParsersFormal Aspects of Component Software10.1007/978-3-030-40914-2_9(170-190)Online publication date: 13-Feb-2020
https://doi.org/10.1007/978-3-030-40914-2_9
Leung ASarracino JLerner S(2015)Interactive parser synthesis by exampleACM SIGPLAN Notices10.1145/2813885.273800250:6(565-574)Online publication date: 3-Jun-2015
https://dl.acm.org/doi/10.1145/2813885.2738002
Leung ASarracino JLerner SGrove DBlackburn S(2015)Interactive parser synthesis by exampleProceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation10.1145/2737924.2738002(565-574)Online publication date: 3-Jun-2015
https://dl.acm.org/doi/10.1145/2737924.2738002
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

PADS: a domain-specific language for processing ad hoc data

PADS: a domain-specific language for processing ad hoc data

PADS/ML: a functional data description language

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations