research-article

NVM: Is it Not Very Meaningful for Databases?

Authors:

Dimitrios Koutsoukos,

Raghav Bhartia,

Michal Friedman,

Gustavo AlonsoAuthors Info & Claims

Proceedings of the VLDB Endowment, Volume 16, Issue 10

Pages 2444 - 2457

https://doi.org/10.14778/3603581.3603586

Published: 01 June 2023 Publication History

Abstract

Persistent or Non Volatile Memory (PMEM) offers expanded memory capacity and faster access to persistent storage. However, there is no comprehensive empirical analysis of existing database engines under different PMEM modes, to understand how databases can benefit from the various hardware configurations. To this end, we analyze multiple different engines under common benchmarks with PMEM in AppDirect mode and Memory mode. Our results show that PMEM in Memory mode does not offer any clear performance advantage despite the larger volatile memory capacity. Also, using PMEM as persistent storage usually speeds up query execution, but with some caveats as the I/O path is not fully optimized and therefore does not always justify the additional cost. We show this to be the case through a comprehensive evaluation of different engines and database configurations under different workloads.

References

[1]

Accessed 2021-10-18. Peloton. https://pelotondb.io/.

[2]

Accessed 2021-10-19. Intel®Optane©DC Persistent Memory. https://www.intel.com/content/www/us/en/architecture-and-technology/optane-dc-persistent-memory.html.

[3]

Accessed 2021-10-25. Intel Vtune Platform Profiler. https://www.intel.com/content/dam/develop/external/us/en/documents/vtuneplatformprofilerwhitepaper.pdf.

[4]

Accessed 2021-11-01. Faster access to more data. https://www.intel.ca/content/www/ca/en/architecture-and-technology/optane-technology/faster-access-to-more-data-article-brief.html.

[5]

Accessed 2021-11-02. InnoDB: Parallel read of index. https://dev.mysql.com/worklog/task/?id=11720#tabs-11720-2.

[6]

Accessed 2021-11-04. PGTune. https://pgtune.leopard.in.ua/.

[7]

Accessed 2021-11-09. Fsync performance on storage devices. https://www.percona.com/blog/2018/02/08/fsync-performance-storage-devices/.

[8]

Accessed 2021-11-09. Mysql 8.0 reference manual: doublewrite buffer. https://dev.mysql.com/doc/refman/8.0/en/innodb-doublewrite-buffer.html.

[9]

Accessed 2021-11-09. Mysql 8.0 reference manual: innoDB startup options and system variables. https://dev.mysql.com/doc/refman/8.0/en/innodb-parameters.html#sysvar_innodb_flush_method.

[10]

Accessed 2021-11-16. fio - Flexible I/O tester rev. 3.27. https://fio.readthedocs.io/en/latest/fio_doc.html.

[11]

Accessed 2021-11-16. Ioping. https://github.com/koct9i/ioping.

[12]

Accessed 2021-11-23. Configure persistent memory (PMEM) for SQL Server on Linux. https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-configure-pmem?view=sql-server-ver15.

[13]

Accessed 2021-11-25. Performance best practices and configuration guidelines for SQL Server on Linux. https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-performance-best-practices?view=sql-server-ver15.

[14]

Accessed 2022-11-22. Compute Express Link. https://www.computeexpresslink.org/.

[15]

Accessed 2022-11-22. Discontinuation of Intel Optane. https://www.intel.com/content/www/us/en/support/products/99743/memory-and-storage/intel-optane-memory/intel-optane-memory-series.html.

[16]

Accessed 2023-03-22. HammerDB. https://www.hammerdb.com/.

[17]

Accessed 2023-03-22. VoltDB-TPCC. https://github.com/VoltDB/voltdb/tree/master/tests/test_apps/tpcc.

[18]

Accessed 24-03-2023. How Does the DRAM Caching Work in Memory Mode Using Intel® Optane™ Persistent Memory? https://www.intel.com/content/www/us/en/support/articles/000055901/memory-and-storage/intel-optane-persistent-memory.html.

[19]

Accessed 24-03-2023. Intel® Optane™ Persistent Memory startup guide. https://www.intel.com/content/dam/support/us/en/documents/memory-and-storage/data-center-persistent-mem/Intel_Optane_Persistent_Memory_Start_Up_Guide.pdf.

[20]

Accessed 24-03-2023. Intel® Optane™ Persistent Memory startup guide. https://www.intel.com/content/dam/support/us/en/documents/memory-and-storage/data-center-persistent-mem/Intel_Optane_Persistent_Memory_Start_Up_Guide.pdf.

[21]

Mihnea Andrei, Christian Lemke, Günter Radestock, Robert Schulze, Carsten Thiel, Rolando Blanco, Akanksha Meghlan, Muhammad Sharique, Sebastian Seifert, Surendra Vishnoi, et al. 2017. SAP HANA adoption of non-volatile memory. Proceedings of the VLDB Endowment (2017), 1754--1765.

Digital Library

[22]

Joy Arulraj and Andrew Pavlo. 2017. How to build a non-volatile memory database management system. In Proceedings of the 2017 ACM International Conference on Management of Data. 1753--1758.

Digital Library

[23]

Joy Arulraj and Andrew Pavlo. 2019. Non-volatile memory database management systems. Synthesis Lectures on Data Management (2019), 1--191.

Digital Library

[24]

Joy Arulraj, Andrew Pavlo, and Subramanya R Dulloor. 2015. Let's talk about storage & recovery methods for non-volatile memory database systems. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data. 707--722.

Digital Library

[25]

Joy Arulraj, Andy Pavlo, and Krishna Teja Malladi. 2019. Multi-tier buffer management and storage system design for non-volatile memory. arXiv preprint arXiv:1901.10938 (2019).

[26]

Joy Arulraj, Matthew Perron, and Andrew Pavlo. 2016. Write-behind logging. Proceedings of the VLDB Endowment (2016), 337--348.

Digital Library

[27]

Lawrence Benson, Hendrik Makait, and Tilmann Rabl. 2021. Viper: An efficient hybrid pmem-dram key-value store. Proceedings of the VLDB Endowment (2021), 1544--1556.

Digital Library

[28]

Lawrence Benson, Leon Papke, and Tilmann Rabl. 2022. PerMA-bench: benchmarking persistent memory access. Proceedings of the VLDB Endowment (2022), 2463--2476.

Digital Library

[29]

Maximilian Böther, Otto Kißig, Lawrence Benson, and Tilmann Rabl. 2021. Drop It In Like It's Hot: An Analysis of Persistent Memory as a Drop-in Replacement for NVMe SSDs. In Proceedings of the 17th International Workshop on Data Management on New Hardware (DaMoN 2021). 1--8.

Digital Library

[30]

Andreas Chatzistergiou, Marcelo Cintra, and Stratis D Viglas. 2015. Rewind: Recovery write-ahead system for in-memory non-volatile data-structures. Proceedings of the VLDB Endowment (2015), 497--508.

[31]

Shimin Chen and Qin Jin. 2015. Persistent b+-trees in non-volatile main memory. Proceedings of the VLDB Endowment (2015), 786--797.

Digital Library

[32]

Youmin Chen, Youyou Lu, Kedong Fang, Qing Wang, and Jiwu Shu. 2020. UTree: A Persistent B+-Tree with Low Tail Latency. Proceedings of the VLDB Endowment (2020), 2634--2648.

Digital Library

[33]

Brian F Cooper, Adam Silberstein, Erwin Tam, Raghu Ramakrishnan, and Russell Sears. 2010. Benchmarking cloud serving systems with YCSB. In Proceedings of the 1st ACM symposium on Cloud computing. 143--154.

Digital Library

[34]

Björn Daase, Lars Jonas Bollmeier, Lawrence Benson, and Tilmann Rabl. 2021. Maximizing persistent memory bandwidth utilization for OLAP workloads. In Proceedings of the 2021 International Conference on Management of Data. 339--351.

Digital Library

[35]

Tudor David, Aleksandar Dragojević, Rachid Guerraoui, and Igor Zablotchi. 2018. Log-Free Concurrent Data Structures. In Proceedings of the 2018 USENIX Conference on Usenix Annual Technical Conference (ATC 2018). 373--385.

Digital Library

[36]

Subramanya R Dulloor, Sanjay Kumar, Anil Keshavamurthy, Philip Lantz, Dheeraj Reddy, Rajesh Sankaran, and Jeff Jackson. 2014. System software for persistent memory. In Proceedings of the Ninth European Conference on Computer Systems. 1--15.

Digital Library

[37]

Michal Friedman, Naama Ben-David, Yuanhao Wei, Guy E. Blelloch, and Erez Petrank. 2020. NVTraverse: In NVRAM Data Structures, the Destination is More Important than the Journey. In Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2020). 377--392.

Digital Library

[38]

Michal Friedman, Erez Petrank, and Pedro Ramalhete. 2021. Mirror: Making Lock-Free Data Structures Persistent. 1218--1232.

[39]

Caixin Gong, Chengjin Tian, Zhengheng Wang, Sheng Wang, Xiyu Wang, Qiulei Fu, Wu Qin, Long Qian, Rui Chen, Jiang Qi, et al. 2022. Tair-PMem: a fully durable non-volatile memory database. Proceedings of the VLDB Endowment (2022), 3346--3358.

Digital Library

[40]

Yuliang He, Duo Lu, Kaisong Huang, and Tianzheng Wang. 2022. Evaluating Persistent Memory Range Indexes: Part Two. arXiv preprint arXiv:2201.13047 (2022).

[41]

Deukyeon Hwang, Wook-Hee Kim, Youjip Won, and Beomseok Nam. 2018. Endurable Transient Inconsistency in Byte-Addressable Persistent B+-Tree. In 16th USENIX Conference on File and Storage Technologies (FAST 18). 187--200.

[42]

Joseph Izraelevitz, Jian Yang, Lu Zhang, Juno Kim, Xiao Liu, Amirsaman Memaripour, Yun Joon Soh, Zixuan Wang, Yi Xu, Subramanya R Dulloor, et al. 2019. Basic performance measurements of the intel optane DC persistent memory module. arXiv preprint arXiv:1903.05714 (2019).

[43]

Andres Lagar-Cavilla, Junwhan Ahn, Suleiman Souhlal, Neha Agarwal, Radoslaw Burny, Shakeel Butt, Jichuan Chang, Ashwin Chaugule, Nan Deng, Junaid Shahid, et al. 2019. Software-defined far memory in warehouse-scale computers. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems. 317--330.

Digital Library

[44]

Lucas Lersch, Xiangpeng Hao, Ismail Oukid, Tianzheng Wang, and Thomas Willhalm. 2019. Evaluating persistent memory range indexes. Proceedings of the VLDB Endowment (2019), 574--587.

Digital Library

[45]

Gang Liu, Leying Chen, and Shimin Chen. 2021. Zen: a high-throughput log-free OLTP engine for non-volatile main memory. Proceedings of the VLDB Endowment (2021), 835--848.

Digital Library

[46]

Baotong Lu, Xiangpeng Hao, Tianzheng Wang, and Eric Lo. 2020. Dash: Scalable hashing on persistent memory. arXiv preprint arXiv:2003.07302 (2020).

[47]

Faisal Nawab, Joseph Izraelevitz, Terence Kelly, Charles B. Morrey, Dhruva R. Chakrabarti, and Michael L. Scott. 2017. Dalí: A Periodically Persistent Hash Map. In 31st International Symposium on Distributed Computing (DISC 2017). 37:1--37:16.

[48]

Onkar Patil, Latchesar Ionkov, Jason Lee, Frank Mueller, and Michael Lang. 2019. Performance characterization of a dram-nvm hybrid memory architecture for hpc applications using intel optane dc persistent memory modules. In Proceedings of the International Symposium on Memory Systems. 288--303.

Digital Library

[49]

Andrew Pavlo, Gustavo Angulo, Joy Arulraj, Haibin Lin, Jiexi Lin, Lin Ma, Prashanth Menon, Todd C Mowry, Matthew Perron, Ian Quah, et al. 2017. Self-Driving Database Management Systems. In CIDR, Vol. 4. 1.

[50]

Ivy B Peng, Maya B Gokhale, and Eric W Green. 2019. System evaluation of the intel optane byte-addressable nvm. In Proceedings of the International Symposium on Memory Systems. 304--315.

Digital Library

[51]

Georgios Psaropoulos, Ismail Oukid, Thomas Legler, Norman May, and Anastasia Ailamaki. 2019. Bridging the latency gap between NVM and DRAM for latency-bound operations. In Proceedings of the 15th International Workshop on Data Management on New Hardware. 1--8.

Digital Library

[52]

Anil Shanbhag, Nesime Tatbul, David Cohen, and Samuel Madden. 2020. Large-scale in-memory analytics on Intel® Optane™ DC persistent memory. In Proceedings of the 16th International Workshop on Data Management on New Hardware. 1--8.

Digital Library

[53]

Alexander van Renen, Viktor Leis, Alfons Kemper, Thomas Neumann, Takushi Hashida, Kazuichi Oe, Yoshiyasu Doi, Lilian Harada, and Mitsuru Sato. 2018. Managing non-volatile memory in database systems. In Proceedings of the 2018 International Conference on Management of Data. 1541--1555.

Digital Library

[54]

Alexander Van Renen, Lukas Vogel, Viktor Leis, Thomas Neumann, and Alfons Kemper. 2019. Persistent memory I/O primitives. In Proceedings of the 15th International Workshop on Data Management on New Hardware. 1--7.

Digital Library

[55]

Shivaram Venkataraman, Niraj Tolia, Parthasarathy Ranganathan, and Roy H. Campbell. 2011. Consistent and Durable Data Structures for Non-Volatile Byte-Addressable Memory. In Proceedings of the 9th USENIX Conference on File and Stroage Technologies (FAST 2011). 5.

[56]

Tianzheng Wang and Ryan Johnson. 2014. Scalable logging through emerging non-volatile memory. Proceedings of the VLDB Endowment (2014).

Digital Library

[57]

Michèle Weiland, Holger Brunst, Tiago Quintino, Nick Johnson, Olivier Iffrig, Simon Smart, Christian Herold, Antonino Bonanni, Adrian Jackson, and Mark Parsons. 2019. An early evaluation of Intel's optane DC persistent memory module and its impact on high-performance scientific applications. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. 1--19.

Digital Library

[58]

Kan Wu, Andrea Arpaci-Dusseau, Remzi Arpaci-Dusseau, Rathijit Sen, and Kwanghyun Park. 2019. Exploiting intel optane ssd for microsoft sql server. In Proceedings of the 15th International Workshop on Data Management on New Hardware. 1--3.

Digital Library

[59]

Yinjun Wu, Kwanghyun Park, Rathijit Sen, Brian Kroth, and Jaeyoung Do. 2020. Lessons learned from the early performance evaluation of Intel Optane DC Persistent Memory in DBMS. In Proceedings of the 16th International Workshop on Data Management on New Hardware. 1--3.

Digital Library

[60]

Lingfeng Xiang, Xingsheng Zhao, Jia Rao, Song Jiang, and Hong Jiang. 2022. Characterizing the performance of intel optane persistent memory: a close look at its on-DIMM buffering. In Proceedings of the Seventeenth European Conference on Computer Systems. 488--505.

Digital Library

[61]

Jian Yang, Juno Kim, Morteza Hoseinzadeh, Joseph Izraelevitz, and Steve Swanson. 2020. An empirical guide to the behavior and use of scalable persistent memory. In 18th {USENIX} Conference on File and Storage Technologies ({FAST} 20). 169--182.

[62]

Jun Yang, Qingsong Wei, Cheng Chen, Chundong Wang, Khai Leong Yong, and Bingsheng He. 2015. NV-Tree: Reducing Consistency Cost for NVM-Based Single Level Systems. In Proceedings of the 13th USENIX Conference on File and Storage Technologies (FAST 2015). 167--181.

[63]

Zhou Zhang, Zhaole Chu, Peiquan Jin, Yongping Luo, Xike Xie, Shouhong Wan, Yun Luo, Xufei Wu, Peng Zou, Chunyang Zheng, Guoan Wu, and Andy Rudoff. 2022. PLIN: A Persistent Learned Index for Non-Volatile Memory with High Performance and Instant Recovery. Proceedings of the VLDB Endowment (2022), 243--255.

Digital Library

[64]

Xinjing Zhou, Joy Arulraj, Andrew Pavlo, and David Cohen. 2021. Spitfire: A Three-Tier Buffer Manager for Volatile and Non-Volatile Memory. In Proceedings of the 2021 International Conference on Management of Data. 2195--2207.

Digital Library

[65]

Yoav Zuriel, Michal Friedman, Gali Sheffi, Nachshon Cohen, and Erez Petrank. 2019. Efficient Lock-Free Durable Sets. In Proceedings of the ACM on Programming Languages (PACMPL 2019).

Digital Library

Cited By

Zheng YTan K(2024)Sorting on Byte-Addressable Storage: The Resurgence of Tree StructureProceedings of the VLDB Endowment10.14778/3648160.364818517:6(1487-1500)Online publication date: 3-May-2024
https://dl.acm.org/doi/10.14778/3648160.3648185
Chu ZJin PLuo YWang XWan S(2024)NOBtree: A NUMA-Optimized Tree Index for Nonvolatile MemoryIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2024.343811143:11(3840-3851)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1109/TCAD.2024.3438111
Miemietz TReusch VRoitzsch MHärtig H(2023)An NVM Performance Study Towards Whole System Persistence on Server PlatformsProceedings of the 1st Workshop on Disruptive Memory Systems10.1145/3609308.3625269(45-51)Online publication date: 23-Oct-2023
https://dl.acm.org/doi/10.1145/3609308.3625269

Recommendations

NVM duet: unified working memory and persistent store architecture
ASPLOS '14: Proceedings of the 19th international conference on Architectural support for programming languages and operating systems

Emerging non-volatile memory (NVM) technologies have gained a lot of attention recently. The byte-addressability and high density of NVM enable computer architects to build large-scale main memory systems. NVM has also been shown to be a promising ...
NVM duet: unified working memory and persistent store architecture
ASPLOS '14

Emerging non-volatile memory (NVM) technologies have gained a lot of attention recently. The byte-addressability and high density of NVM enable computer architects to build large-scale main memory systems. NVM has also been shown to be a promising ...
NVM duet: unified working memory and persistent store architecture
ASPLOS '14

Emerging non-volatile memory (NVM) technologies have gained a lot of attention recently. The byte-addressability and high density of NVM enable computer architects to build large-scale main memory systems. NVM has also been shown to be a promising ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the VLDB Endowment

Proceedings of the VLDB Endowment Volume 16, Issue 10

June 2023

295 pages

ISSN:2150-8097

Editors:
Georgia Koutrika
Athena Research Center
,
Jun Yang
Duke University

Issue’s Table of Contents

Publisher

VLDB Endowment

Publication History

Published: 01 June 2023

Published in PVLDB Volume 16, Issue 10

Check for updates

Badges

Artifacts Available / v1.1

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
184
Total Downloads

Downloads (Last 12 months)63
Downloads (Last 6 weeks)7

Reflects downloads up to 28 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zheng YTan K(2024)Sorting on Byte-Addressable Storage: The Resurgence of Tree StructureProceedings of the VLDB Endowment10.14778/3648160.364818517:6(1487-1500)Online publication date: 3-May-2024
https://dl.acm.org/doi/10.14778/3648160.3648185
Chu ZJin PLuo YWang XWan S(2024)NOBtree: A NUMA-Optimized Tree Index for Nonvolatile MemoryIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2024.343811143:11(3840-3851)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1109/TCAD.2024.3438111
Miemietz TReusch VRoitzsch MHärtig H(2023)An NVM Performance Study Towards Whole System Persistence on Server PlatformsProceedings of the 1st Workshop on Disruptive Memory Systems10.1145/3609308.3625269(45-51)Online publication date: 23-Oct-2023
https://dl.acm.org/doi/10.1145/3609308.3625269

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents