article

Free access

A query language for multidimensional arrays: design, implementation, and optimization techniques

Authors:

Limsoon WongAuthors Info & Claims

ACM SIGMOD Record, Volume 25, Issue 2

Pages 228 - 239

https://doi.org/10.1145/235968.233335

Published: 01 June 1996 Publication History

Abstract

While much recent research has focussed on extending databases beyond the traditional relational model, relatively little has been done to develop database tools for querying data organized in (multidimensional) arrays. The scientific computing community has made little use of available database technology. Instead, multidimensional scientific data is typically stored in local files conforming to various data exchange formats and queried via specialized access libraries tied in to general purpose programming languages.To allow such data to be queried using known database techniques, we design and implement a query language for multidimensional arrays. Our main design decision is to treat arrays as functions from index sets to values rather than as collection types. This leads to clean syntax and semantics as well as simple but powerful optimization rules.We present a calculus for arrays that extends standard calculi for complex objects. We derive a higher-level comprehension style query language based on this calculus and describe its implementation, including a data driver for the NetCDF data exchange format. Next, we explore some optimization rules obtained from the equational laws of our core calculus. Finally, we study the expressiveness of our calculus and prove that it essentially corresponds to adding ranking to a query language for complex objects.

References

[1]

Arvind, R.S. Nikhil, and K.K. Pingali. i-structures: Data structures for parallel computing. A CM Trans. Progr. Lang. Syst. 11 (1989), 598-632.]]

Digital Library

[2]

H. Barendregt. Lambda Calculus: Its Syntaz and Semantics. North Holland, 1984.]]

[3]

C. Beeri and D.K.C. Chan. Bounded arrays: a bulk type perspective. Hebrew Univ. Technical Report, 1995.]]

[4]

P. Buneman. The fast Fourier transform as a database query. Technical Report MS-CIS-93-37/L&C 60, University of Pennsylvania, March 1993.]]

[5]

P. Buneman, S. Davidson, K. Hart, C. Overton, L. Wong. A data transformation system for biological data sources. In VLDB'95, pages 158-169.]]

Digital Library

[6]

P. Buneman, L. Libkin, D. Suciu, V. Tannen, and Lo Wong. Comprehension syntax. SIGMOD Record, 23(1):87-96, March 1994o]]

Digital Library

[7]

P. Buneman, S. Naqvi~ V. Tannen and L. Wong. Principles of programming with complex objects and collection types. Theor. Comp. Sci., 149 (1995), 3-48.]]

Digital Library

[8]

R.G.G. Cattell, ed. The Object Database Standard: ODMG-93. Morgan-Kaufmann, 1994.]]

[9]

L. Fegaras and D. Maier. Towards an effective calculus for object query languages. In SIGMOD'95, pages 47- 58.]]

Digital Library

[10]

J. Feo. Arrays in Sisal. in Proc. Workshop on Arrays, Functional Languages and Parallel Systems, L. Mullin et al. eds., Kluwer Academic PubLishers, 1990.]]

[11]

S. Greco, P~ Palopoli and E. Spadafora. Dataloga: Array manipulations in a deductive database language. In Proc.,~th Conf. on Database Systems for Advanced Applieations, pages 180-188, 1995.]]

Digital Library

[12]

D. Gries. The Science of Programming. Springer- Ver|ag, 1981.]]

Digital Library

[13]

S. Grumbach and T~ Milo. Towards tractable algebras for bags. In PODS'93, pages 49-58.]]

Digital Library

[14]

P. Hammarlund and B. Lisper. On the relation between functional and data parallel programming languages. In FPCA '93, pages 9.10-219.]]

Digital Library

[15]

P. Hudak, S.L. Peyton Jones and P. Wadler. Report on the Programming Language HaskeU. SIGPLAN Notices, March 1992.]]

Digital Library

[16]

K. E. Iverson. A Programming Language. Wiley, 1962.]]

Digital Library

[17]

T. W. Leung, B. Subramaniam, S. Vandenberg and S. B. Zdonik. Ordered types in the AQUA data model. In DBPL'93, pages 115-135.]]

Digital Library

[18]

L. Libkin and L. Wong. Aggregate functions, conservative extensions, and linear orders. In DBPL'93, pages 282-294.]]

Digital Library

[19]

L. Libkin and L. Wong. Some properties of query languages for bags. In DBPL'93, pages 97-114.]]

Digital Library

[20]

L. Libkin and L. Wong. New techniques for studying set languages, bag languages and aggregate functions. In PODS'9~, pages 155-166.]]

[21]

L. Libkin and L. Wong. Conservativity of nested relational calculi with internal generic functions. Information Processing Letters, 49(6):273-280, March 1994.]]

Digital Library

[22]

D. Maier and B. Vance. A call to order. In PODS'93, pages 1-1fi.]]

Digital Library

[23]

D. Maier and D. Hansen. Bambi meets Godzilla: Object databases for scientific computing, in Proe. 7th Working Conference on ~qclentifie and Statistical Database Management, 1994, pages 176-184.]]

Digital Library

[24]

G. Mecca and A. Bormer. Sequences, datalog and transducers. In PODS'95, pages 23-35.]]

Digital Library

[25]

G. Mecca and A. Bonner. Finite query languages for sequence databases. In DBPL '95, to appear.]]

Digital Library

[26]

Milner, M. Torte, R. Harper. "The Definition of Standard ML ". The MIT Press, Cambridge, Mass, 1990.]]

Digital Library

[27]

T. More. Axioms and theorems for a theory of arrays. IBM J. Res. and Development 17 (1973), 135-175.]]

Digital Library

[28]

R. Rew, G. Davis and S. Emmerson. NetGDF User's Guide, Unidata Program Center, 1993.]]

[29]

P. Seshadri, M. Livny and R.Ramakrishnan. Sequence query processing. In SIGMOD'94, pages 430-441.]]

Digital Library

[30]

P. Seshadri, M. Livny and R.Ramakrishnan. ~q~Q: a model for sequence databases. In ICDE'95, pages 232- 239.]]

Digital Library

[31]

So Vandenberg. Algebras .for Object-Oriented Query Languages. PhD thesis, Univ. of Wisconsin, 1993.]]

Digital Library

[32]

S. Vandenberg and D. DeWitt. Algebraic support for complex objects with arrays, identity and inheritance. In SIGMOD'91, pages 158-167.]]

Digital Library

[33]

P. Wadler. Comprehending monads. Mathematical Structures in Computer Science 2 (1992), 461-493.]]

[34]

L. Wong. Querying Nested Collections. PhD thesis, Univ. of Pennsylvania, August 1994.]]

Digital Library

Cited By

Rodriges Zalipynis R(2024)Quantum Tensor DBMS and Quantum Gantt Charts: Towards Exponentially Faster Earth Data EngineeringEarth10.3390/earth50300275:3(491-547)Online publication date: 14-Sep-2024
https://doi.org/10.3390/earth5030027
Reznichenko V(2022)60 Years of Databases (part three)PROBLEMS IN PROGRAMMING10.15407/pp2022.01.034(034-066)Online publication date: Mar-2022
https://doi.org/10.15407/pp2022.01.034
Song JAlter GJagadish H(2022)Structured data transformation algebra (SDTA) and its applicationsDistributed and Parallel Databases10.1007/s10619-022-07418-640:2-3(373-408)Online publication date: 1-Sep-2022
https://dl.acm.org/doi/10.1007/s10619-022-07418-6
Show More Cited By

Index Terms

A query language for multidimensional arrays: design, implementation, and optimization techniques
1. Information systems
  1. Data management systems
    1. Query languages

Recommendations

A query language for multidimensional arrays: design, implementation, and optimization techniques
SIGMOD '96: Proceedings of the 1996 ACM SIGMOD international conference on Management of data

While much recent research has focussed on extending databases beyond the traditional relational model, relatively little has been done to develop database tools for querying data organized in (multidimensional) arrays. The scientific computing ...
Multidimensional data model and query language for informetrics

Multidimensional data analysis or On-line analytical processing (OLAP) offers a single subject-oriented source for analyzing summary data based on various dimensions. We demonstrate that the OLAP approach gives a promising starting point for advanced ...
Query XML Documents Using XTQ Language
WSCS '08: Proceedings of the IEEE International Workshop on Semantic Computing and Systems

Up to now many XML query languages, including XPath and XQuery which become the standard of XML query standard by W3C, have been prosposed. However, since the navigational query approach adopted by XPath can only get homogeneous data, it often incurs ...

Comments

Information & Contributors

Information

Published In

cover image ACM SIGMOD Record

ACM SIGMOD Record Volume 25, Issue 2

June 1996

557 pages

ISSN:0163-5808

DOI:10.1145/235968

Chairman:
T. H. Merrett
McGill Univ.
,
Editors:
H. V. Jagadish,
Inderpal Singh Mumick

Issue’s Table of Contents

SIGMOD '96: Proceedings of the 1996 ACM SIGMOD international conference on Management of data
June 1996
560 pages
ISBN:0897917944
DOI:10.1145/233269
Editor:
Jennifer Widom
Stanford Univ., Stanford, CT

Copyright © 1996 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 June 1996

Published in SIGMOD Volume 25, Issue 2

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

82
Total Citations
View Citations
932
Total Downloads

Downloads (Last 12 months)109
Downloads (Last 6 weeks)36

Reflects downloads up to 29 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Rodriges Zalipynis R(2024)Quantum Tensor DBMS and Quantum Gantt Charts: Towards Exponentially Faster Earth Data EngineeringEarth10.3390/earth50300275:3(491-547)Online publication date: 14-Sep-2024
https://doi.org/10.3390/earth5030027
Reznichenko V(2022)60 Years of Databases (part three)PROBLEMS IN PROGRAMMING10.15407/pp2022.01.034(034-066)Online publication date: Mar-2022
https://doi.org/10.15407/pp2022.01.034
Song JAlter GJagadish H(2022)Structured data transformation algebra (SDTA) and its applicationsDistributed and Parallel Databases10.1007/s10619-022-07418-640:2-3(373-408)Online publication date: 1-Sep-2022
https://dl.acm.org/doi/10.1007/s10619-022-07418-6
Zalipynis R(2021)Array DBMSProceedings of the VLDB Endowment10.14778/3476311.347640414:12(3186-3189)Online publication date: 28-Oct-2021
https://dl.acm.org/doi/10.14778/3476311.3476404
Zalipynis R(2018)ChronosDBProceedings of the VLDB Endowment10.14778/3231751.323175411:10(1247-1261)Online publication date: 1-Jun-2018
https://dl.acm.org/doi/10.14778/3231751.3231754
Ebenstein RAgrawal GWang JBoley JKettimuthu R(2018)FDQ: Advance Analytics Over Real Scientific Array Datasets2018 IEEE 14th International Conference on e-Science (e-Science)10.1109/eScience.2018.00134(453-463)Online publication date: Oct-2018
https://doi.org/10.1109/eScience.2018.00134
Xing HFloratos SBlanas SByna SPrabhat MWu KBrown P(2018)ArrayBridge: Interweaving Declarative Array Processing in SciDB with Imperative HDF5-Based Programs2018 IEEE 34th International Conference on Data Engineering (ICDE)10.1109/ICDE.2018.00092(977-988)Online publication date: Apr-2018
https://doi.org/10.1109/ICDE.2018.00092
Baumann P(2018)Array DatabasesEncyclopedia of Database Systems10.1007/978-1-4614-8265-9_2061(165-177)Online publication date: 7-Dec-2018
https://doi.org/10.1007/978-1-4614-8265-9_2061
Gao ZLuo SPerez LJermaine CChirkova RYang JSuciu D(2017)The BUDS Language for Distributed Bayesian Machine LearningProceedings of the 2017 ACM International Conference on Management of Data10.1145/3035918.3035937(961-976)Online publication date: 9-May-2017
https://dl.acm.org/doi/10.1145/3035918.3035937
Luo SGao ZGubanov MPerez LJermaine C(2017)Scalable Linear Algebra on a Relational Database System2017 IEEE 33rd International Conference on Data Engineering (ICDE)10.1109/ICDE.2017.108(523-534)Online publication date: Apr-2017
https://doi.org/10.1109/ICDE.2017.108
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Issue’s Table of Contents