research-article

Vectorization vs. compilation in query execution

Authors:

Juliusz Sompolski,

Marcin Zukowski,

Peter BonczAuthors Info & Claims

DaMoN '11: Proceedings of the Seventh International Workshop on Data Management on New Hardware

Pages 33 - 40

https://doi.org/10.1145/1995441.1995446

Published: 13 June 2011 Publication History

Get Access

Abstract

Compiling database queries into executable (sub-) programs provides substantial benefits comparing to traditional interpreted execution. Many of these benefits, such as reduced interpretation overhead, better instruction code locality, and providing opportunities to use SIMD instructions, have previously been provided by redesigning query processors to use a vectorized execution model. In this paper, we try to shed light on the question of how state-of-the-art compilation strategies relate to vectorized execution for analytical database workloads on modern CPUs. For this purpose, we carefully investigate the behavior of vectorized and compiled strategies inside the Ingres VectorWise database system in three use cases: Project, Select and Hash Join. One of the findings is that compilation should always be combined with block-wise query execution. Another contribution is identifying three cases where "loop-compilation" strategies are inferior to vectorized execution. As such, a careful merging of these two strategies is proposed for optimal performance: either by incorporating vectorized execution principles into compiled query plans or using query compilation to create building blocks for vectorized processing.

References

[1]

P. Boncz, M. Zukowski, and N. Nes. MonetDB/X100: Hyper-Pipelining Query Execution. In Proc. CIDR, Asilomar, CA, USA, 2005.

Google Scholar

[2]

P. A. Boncz. Monet: A Next-Generation DBMS Kernel For Query-Intensive Applications. Ph.d. thesis, Universiteit van Amsterdam, Amsterdam, The Netherlands, May 2002.

Google Scholar

[3]

S. Chen, A. Ailamaki, P. B. Gibbons, and T. C. Mowry. Improving hash join performance through prefetching. In Proc. ICDE, Boston, MA, USA, 2004.

Digital Library

Google Scholar

[4]

D. Chamberlin et al. A history and evaluation of System R. Commun. ACM, 24(10):632--646, 1981.

Digital Library

Google Scholar

[5]

G. Graefe. Volcano - an extensible and parallel query evaluation system. IEEE TKDE, 6(1):120--135, 1994.

Digital Library

Google Scholar

[6]

A. Kemper and T. Neumann. HyPer: Hybrid OLTP and OLAP High Performance Database System. Technical report, Technical Univ. Munich, TUM-I1010, May 2010.

Google Scholar

[7]

K. Krikellas, S. Viglas, and M. Cintra. Generating code for holistic query evaluation. In ICDE, pages 613--624, 2010.

Crossref

Google Scholar

[8]

S. Padmanabhan, T. Malkemus, R. Agarwal, and A. Jhingran. Block Oriented Processing of Relational Database Operations in Modern Computer Architectures. In Proc. ICDE, Heidelberg, Germany, 2001.

Digital Library

Google Scholar

[9]

ParAccel Inc. Whitepaper. The ParAcel Analytical Database: A Technical Overview, Feb 2010. http://www.paraccel.com.

Google Scholar

[10]

J. Rao, H. Pirahesh, C. Mohan, and G. M. Lohman. Compiled Query Execution Engine using JVM. In Proc. ICDE, Atlanta, GA, USA, 2006.

Digital Library

Google Scholar

[11]

K. A. Ross. Conjunctive selection conditions in main memory. In Proc. PODS, Washington, DC, USA, 2002.

Digital Library

Google Scholar

[12]

The LLVM Compiler Infrastructure. http://llvm.org.

Google Scholar

[13]

M. Zukowski. Balancing Vectorized Query Execution with Bandwidth-Optimized Storage. Ph.D. Thesis, Universiteit van Amsterdam, Sep 2009.

Google Scholar

[14]

M. Zukowski, N. Nes, and P. Boncz. DSM vs. NSM: CPU Performance Tradeoffs in Block-Oriented Query Processing. 2008.

Google Scholar

Cited By

View all

Kozar ABleichert JBreß SGrulich PLutz CRabl TRosenfeld VTraub JZeuch SMarkl V(2025)Query Processing on Heterogeneous HardwareScalable Data Management for Future Hardware10.1007/978-3-031-74097-8_2(39-64)Online publication date: 24-Jan-2025
https://doi.org/10.1007/978-3-031-74097-8_2
Lv YZhang KWang ZZhang XLee RHe ZJing YWang X(2024)RTScan: Efficient Scan with Ray Tracing CoresProceedings of the VLDB Endowment10.14778/3648160.364818317:6(1460-1472)Online publication date: 3-May-2024
https://doi.org/10.14778/3648160.3648183
Baumstark AParadies MSattler KKläbe SBaumann S(2024)So Far and yet so Near - Accelerating Distributed Joins with CXLProceedings of the 20th International Workshop on Data Management on New Hardware10.1145/3662010.3663449(1-9)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3662010.3663449
Show More Cited By

Index Terms

Vectorization vs. compilation in query execution

Recommendations

Outer-loop vectorization: revisited for short SIMD architectures
PACT '08: Proceedings of the 17th international conference on Parallel architectures and compilation techniques

Vectorization has been an important method of using data-level parallelism to accelerate scientific workloads on vector machines such as Cray for the past three decades. In the last decade it has also proven useful for accelerating multi-media and ...
Compile-Time Analysis of Compiler Frameworks for Query Compilation
CGO '24: Proceedings of the 2024 IEEE/ACM International Symposium on Code Generation and Optimization

Low compilation times are highly important in contexts of Just-in-time compilation. This not only applies to language runtimes for Java, WebAssembly, or JavaScript, but is also crucial for database systems that employ query compilation as the primary ...
Mitigating JIT compilation latency in virtual execution environments
VEE 2019: Proceedings of the 15th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments

Many Virtual Execution Environments (VEEs) rely on Just-in-time (JIT) compilation technology for code generation at runtime, e.g. in Dynamic Binary Translation (DBT) systems or language Virtual Machines (VMs). While JIT compilation improves native ...

Comments

Information & Contributors

Information

Published In

DaMoN '11: Proceedings of the Seventh International Workshop on Data Management on New Hardware

June 2011

58 pages

ISBN:9781450306584

DOI:10.1145/1995441

Editors:
Stavros Harizopoulos
HP Labs MAIL@[email protected]
,
Qiong Luo
Hong Kong University of Science and Technology MAIL@[email protected]

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 June 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

SIGMOD/PODS '11

Sponsor:

SIGMOD

SIGMOD/PODS '11: International Conference on Management of Data

June 13, 2011

Athens, Greece

Acceptance Rates

Overall Acceptance Rate 94 of 127 submissions, 74%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

58
Total Citations
View Citations
753
Total Downloads

Downloads (Last 12 months)91
Downloads (Last 6 weeks)5

Reflects downloads up to 28 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Kozar ABleichert JBreß SGrulich PLutz CRabl TRosenfeld VTraub JZeuch SMarkl V(2025)Query Processing on Heterogeneous HardwareScalable Data Management for Future Hardware10.1007/978-3-031-74097-8_2(39-64)Online publication date: 24-Jan-2025
https://doi.org/10.1007/978-3-031-74097-8_2
Lv YZhang KWang ZZhang XLee RHe ZJing YWang X(2024)RTScan: Efficient Scan with Ray Tracing CoresProceedings of the VLDB Endowment10.14778/3648160.364818317:6(1460-1472)Online publication date: 3-May-2024
https://doi.org/10.14778/3648160.3648183
Baumstark AParadies MSattler KKläbe SBaumann S(2024)So Far and yet so Near - Accelerating Distributed Joins with CXLProceedings of the 20th International Workshop on Data Management on New Hardware10.1145/3662010.3663449(1-9)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3662010.3663449
Li JXie QMa YMa JJi KZhang YZhang CChen YWu GZhang JYang KHe XShen QTao YZhao HJiao PZhu CQian DXu C(2023)Big Data Analytic Toolkit: A General-Purpose, Modular, and Heterogeneous Acceleration Toolkit for Data Analytical EnginesProceedings of the VLDB Endowment10.14778/3611540.361155816:12(3702-3714)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.14778/3611540.3611558
Liu ZHan RZhang YZhang YTang XDeng GZhong TDementiev RLu YQue M(2023)Exploring Fine-Grained In-Memory Database Performance for Modern CPUsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2023.3262782(1-16)Online publication date: 2023
https://doi.org/10.1109/TPDS.2023.3262782
Sun ZLi ZWeng C(2023)Co-Utilizing SIMD and Scalar to Accelerate the Data Analytics Workloads2023 IEEE 39th International Conference on Data Engineering (ICDE)10.1109/ICDE55515.2023.00387(637-649)Online publication date: May-2023
https://doi.org/10.1109/ICDE55515.2023.00387
Kuiper LMühleisen H(2023)These Rows Are Made for Sorting and That’s Just What We’ll Do2023 IEEE 39th International Conference on Data Engineering (ICDE)10.1109/ICDE55515.2023.00159(2050-2062)Online publication date: May-2023
https://doi.org/10.1109/ICDE55515.2023.00159
Mostafa JWehbi SChilingaryan SKopmann A(2022)SciTS: A Benchmark for Time-Series Databases in Scientific Experiments and Industrial Internet of ThingsProceedings of the 34th International Conference on Scientific and Statistical Database Management10.1145/3538712.3538723(1-11)Online publication date: 6-Jul-2022
https://dl.acm.org/doi/10.1145/3538712.3538723
Behm APalkar SAgarwal UArmstrong TCashman DDave AGreenstein THovsepian SJohnson RSai Krishnan ALeventis PLuszczak AMenon PMokhtar MPang GParanjpye SRahn GSamwel Bvan Bussel Tvan Hovell HXue MXin RZaharia MIves ZBonifati AEl Abbadi A(2022)Photon: A Fast Query Engine for Lakehouse SystemsProceedings of the 2022 International Conference on Management of Data10.1145/3514221.3526054(2326-2339)Online publication date: 10-Jun-2022
https://dl.acm.org/doi/10.1145/3514221.3526054
Gupta SRamachandra K(2021)Procedural extensions of SQLProceedings of the VLDB Endowment10.14778/3457390.345740214:8(1378-1391)Online publication date: 21-Oct-2021
https://dl.acm.org/doi/10.14778/3457390.3457402
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Outer-loop vectorization: revisited for short SIMD architectures

Compile-Time Analysis of Compiler Frameworks for Query Compilation

Mitigating JIT compilation latency in virtual execution environments

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations