research-article

WATuning: A Workload-Aware Tuning System with Attention-Based Deep Reinforcement Learning

Authors:

Yun-Peng ChaiAuthors Info & Claims

Journal of Computer Science and Technology, Volume 36, Issue 4

Pages 741 - 761

https://doi.org/10.1007/s11390-021-1350-8

Published: 01 July 2021 Publication History

Abstract

Configuration tuning is essential to optimize the performance of systems (e.g., databases, key-value stores). High performance usually indicates high throughput and low latency. At present, most of the tuning tasks of systems are performed artificially (e.g., by database administrators), but it is hard for them to achieve high performance through tuning in various types of systems and in various environments. In recent years, there have been some studies on tuning traditional database systems, but all these methods have some limitations. In this article, we put forward a tuning system based on attention-based deep reinforcement learning named WATuning, which can adapt to the changes of workload characteristics and optimize the system performance efficiently and effectively. Firstly, we design the core algorithm named ATT-Tune for WATuning to achieve the tuning task of systems. The algorithm uses workload characteristics to generate a weight matrix and acts on the internal metrics of systems, and then ATT-Tune uses the internal metrics with weight values assigned to select the appropriate configuration. Secondly, WATuning can generate multiple instance models according to the change of the workload so that it can complete targeted recommendation services for different types of workloads. Finally, WATuning can also dynamically fine-tune itself according to the constantly changing workload in practical applications so that it can better fit to the actual environment to make recommendations. The experimental results show that the throughput and the latency of WATuning are improved by 52.6% and decreased by 31%, respectively, compared with the throughput and the latency of CDBTune which is an existing optimal tuning method.

References

[1]

O’Neil P, Cheng E, Gawlick D, and O’Neil E The log-structured merge-tree (LSM-tree) Acta Informatica 1996 33 4 351-385

[2]

Dong S Y, Callaghan M, Galanis L, Borthakur D, Savor T, Stumm M. Optimizing space amplification in RocksDB. In Proc. the 8th Biennial Conference on Innovative Data Systems Research, Jan. 2017.

[3]

Chai Y P, Chai Y F, Wang X, Wei H C, Bao N, Liang Y S. LDC: A lower-level driven compaction method to optimize SSD-oriented key-value stores. In Proc. the 35th IEEE International Conference on Data Engineering, April 2019, pp.722-733.

[4]

Chai Y P, Chai Y F, Wang X, Wei H C, Wang Y Y. Adaptive lower-level driven compaction to optimize LSM-Tree key-value stores. IEEE Transactions on Knowledge Data Engineering..

[5]

Zhu Y Q, Liu J X, Guo M Y, Bao Y G, Ma W L, Liu Z Y, Song K P, Yang Y C. BestConfig: Tapping the performance potential of systems via automatic configuration tuning. In Proc. ACM Symposium on Cloud Computing, Sept. 2017, pp.338-350.

[6]

Van Aken D, Pavlo A, Gordon G J, Zhang B H. Automatic database management system tuning through large-scale machine learning. In Proc. the 2017 ACM International Conference on Management of Data, May 2017, pp.1009-1024.

[7]

Zhang J, Liu L, Ran M, Li Z K, Liu Y, Zhou K, Li G L, Xiao Z L, Cheng B, Xing J S, Wang Y T, Cheng T H. An end-to-end automatic cloud database tuning system using deep reinforcement learning. In Proc. the 2019 International Conference on Management of Data, June 2019, pp.415-432.

[8]

Li G L, Zhou X H, Li S F, Gao B. QTune: A query-aware database tuning system with deep reinforcement learning. Proc. the VLDB Endowment, 2019, 12(12): 2118-2130.

[9]

Lillicrap T P, Hunt J J, Pritzel A, Heess N, Erez T, Tassa Y, Silver D, Wierstra D. Continuous control with deep reinforcement learning. arXiv:1509.02971, 2015. https://arxiv.org/abs/1509.02971, Jun. 2021.

[10]

Van Hasselt H. Double Q-learning. In Proc. the 24th Annual Conference on Neural Information Processing Systems, Dec. 2010, pp.2613-2621.

[11]

Kingma D, Ba J. Adam: A method for stochastic optimization. In Proc. the 3rd International Conference on Learning Representations, May 2015.

[12]

Munos R and Moore A Variable resolution discretization in optimal control Machine Learning 2002 49 2/3 291-323

[13]

Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning Nature 2015 518 7540 529-533

[14]

Ban TW An autonomous transmission scheme using dueling DQN for D2D communication networks IEEE Transactions on Vehicular Technology 2020 69 12 16348-16352

[15]

Chen L, Hu X M, Tang B, Cheng Y. Conditional DQN-based motion planning with fuzzy logic for autonomous driving. IEEE Transactions on Intelligent Transportation Systems.

[16]

Huang HJ, Yang YC, Wang H, Ding ZG, Sari H, and Adachi F Deep reinforcement learning for UAV navigation through massive MIMO technique IEEE Transactions on Vehicular Technology 2020 69 1 1117-1121

[17]

Li JX, Yao L, Xu X, Cheng B, and Ren JK Deep reinforcement learning for pedestrian collision avoidance and human-machine cooperative driving Information Sciences 2020 532 110-124

[18]

Yoo H, Kim B, Kim J W, Lee J H. Reinforcement learning based optimal control of batch processes using Monte-Carlo deep deterministic policy gradient with phase segmentation. Computers & Chemical Engineering, 2021, 144: Article No. 107133.

[19]

He XM, Lu HD, Du M, Mao YC, and Wang K QoE-based task offloading with deep reinforcement learning in edge-enabled Internet of vehicles IEEE Transactions on Intelligent Transportation Systems 2020 22 4 2252-2261

[20]

Li LY, Xu H, Ma J, and Zhou AZ Joint EH time and transmit power optimization based on DDPG for EH communications IEEE Communications Letters 2020 24 9 2043-2046

[21]

Nguyen DQ, Vien NA, Dang VH, and Chung T Asynchronous framework with Reptile+ algorithm to meta learn partially observable Markov decision process Applied Intelligence 2020 50 11 4050-4062

[22]

Gheisarnejad M and Khooban MH IoT-based DC/DC deep learning power converter control: Real-time implementation IEEE Transactions on Power Electronics 2020 35 12 13621-13630

[23]

Tang Z T, Shao K, Zhao D B, Zhu Y H. Recent progress of deep reinforcement learning: From AlphaGo to AlphaGoZero. Control Theory & Applications, 2017, 34(12): 1529-1546. (in Chinese)

[24]

Silver D, Schrittwieser J, Simonyan K, et al. Mastering the game of Go without human knowledge Nature 2017 550 7676 354-359

[25]

Ye D H, Chen G B, Zhang W et al. Towards playing full MOBA games with deep reinforcement learning. arXiv:2011.12692, 2020. https://arxiv.org/abs/2011.12692, Dec. 2020.

[26]

Li G L. Human-in-the-loop data integration. Proceedings of the VLDB Endowment, 2017, 10(12): 2006-2017.

[27]

Li GL, Zhou XH, and Li SH XuanYuan: An AI-native database IEEE Data Engineering Bulletin 2019 42 2 70-81

[28]

Basu D, Lin Q, Chen W, Vo H T, Yuan Z, Senellart P, Bressan S. Regularized cost-model oblivious database tuning with reinforcement learning. In Transactions on Large-Scale Data- and Knowledge-Centered Systems XXVIII, Hameurlain A, Küng J, Wagner R, Chen Q (eds.), Springer, 2016, pp.96-132.

[29]

Sun J, Li G L. An end-to-end learning-based cost estimator. Proceedings of the VLDB Endowment, 2019, 13(3): 307-319.

[30]

Kraska T, Alizadeh M, Beutel A et al. SageDB: A learned database system. In Proc. the 9th Biennial Conference on Innovative Data Systems Research, Jan. 2019.

[31]

Duan S Y, Thummala V, Babu S. Tuning database configuration parameters with iTuned. Proceedings of the VLDB Endowment, 2009, 2(1): 1246-1257.

[32]

Wei Z J, Ding Z H, Hu J L. Self-tuning performance of database systems based on fuzzy rules. In Proc. the 11th International Conference on Fuzzy Systems and Knowledge Discovery, Aug. 2014, pp.194-198.

[33]

Zheng C H, Ding Z H, Hu J L. Self-tuning performance of database systems with neural network. In Proc. the 10th International Conference on Natural Computation, Aug. 2014, pp.1-12.

Cited By

Bianchi AChai ACorvinelli VGodfrey PSzlichta JZuzarte C(2024)Db2une: Tuning Under Pressure via Deep LearningProceedings of the VLDB Endowment10.14778/3685800.368581117:12(3855-3868)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.14778/3685800.3685811
Mai GHe ZYu GChen ZChen P(2024)CTuner: Automatic NoSQL Database Tuning with Causal Reinforcement LearningProceedings of the 15th Asia-Pacific Symposium on Internetware10.1145/3671016.3674809(269-278)Online publication date: 24-Jul-2024
https://dl.acm.org/doi/10.1145/3671016.3674809
Zhan YXi RLiao JFan SHou M(2024)KnobTune: A Dynamic Database Configuration Tuning Strategy Leveraging Historical Workload SimilaritiesProceedings of the International Conference on Computing, Machine Learning and Data Science10.1145/3661725.3661734(1-8)Online publication date: 12-Apr-2024
https://dl.acm.org/doi/10.1145/3661725.3661734
Show More Cited By

Index Terms

WATuning: A Workload-Aware Tuning System with Attention-Based Deep Reinforcement Learning

Index terms have been assigned to the content through auto-classification.

Recommendations

An End-to-End Automatic Cloud Database Tuning System Using Deep Reinforcement Learning
SIGMOD '19: Proceedings of the 2019 International Conference on Management of Data

Configuration tuning is vital to optimize the performance of database management system (DBMS). It becomes more tedious and urgent for cloud databases (CDB) due to the diverse database instances and query workloads, which make the database administrator ...
Mjolnir: A framework agnostic auto-tuning system with deep reinforcement learning
Abstract
Choosing the right setting for big data frameworks is an important yet difficult task. These frameworks come with a complex set of parameters that need to be tuned to achieve the best performance in terms of throughput and latency. Learning-based ...
${CDBTune}^{+}$ : An efficient deep reinforcement learning-based automatic cloud database tuning system
Abstract
Configuration tuning is vital to optimize the performance of a database management system (DBMS). It becomes more tedious and urgent for cloud databases (CDB) due to diverse database instances and query workloads, which make the job of a database ... $^{}$ $^{}$ $^{}$ $^{}$ $^{}$ $^{}$ $^{}$

Comments

Information & Contributors

Information

Published In

cover image Journal of Computer Science and Technology

Journal of Computer Science and Technology Volume 36, Issue 4

Jul 2021

242 pages

ISSN:1000-9000

Issue’s Table of Contents

© Institute of Computing Technology, Chinese Academy of Sciences 2021.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 01 July 2021

Accepted: 24 June 2021

Received: 01 February 2021

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 28 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bianchi AChai ACorvinelli VGodfrey PSzlichta JZuzarte C(2024)Db2une: Tuning Under Pressure via Deep LearningProceedings of the VLDB Endowment10.14778/3685800.368581117:12(3855-3868)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.14778/3685800.3685811
Mai GHe ZYu GChen ZChen P(2024)CTuner: Automatic NoSQL Database Tuning with Causal Reinforcement LearningProceedings of the 15th Asia-Pacific Symposium on Internetware10.1145/3671016.3674809(269-278)Online publication date: 24-Jul-2024
https://dl.acm.org/doi/10.1145/3671016.3674809
Zhan YXi RLiao JFan SHou M(2024)KnobTune: A Dynamic Database Configuration Tuning Strategy Leveraging Historical Workload SimilaritiesProceedings of the International Conference on Computing, Machine Learning and Data Science10.1145/3661725.3661734(1-8)Online publication date: 12-Apr-2024
https://dl.acm.org/doi/10.1145/3661725.3661734
Dou HWang YZhang YChen PZheng Z(2024)DeepCAT⁺: A Low-Cost and Transferrable Online Configuration Auto-Tuning Approach for Big Data FrameworksIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2024.345988935:11(2114-2131)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1109/TPDS.2024.3459889
Nie XLiu YFu FXue JJiao DMiao XTao YCui B(2023)Angel-PTM: A Scalable and Economical Large-Scale Pre-Training System in TencentProceedings of the VLDB Endowment10.14778/3611540.361156416:12(3781-3794)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.14778/3611540.3611564
Sun JYe FNedjah NZhang MXu D(2023)Workload-Aware Performance Tuning for Multimodel Databases Based on Deep Reinforcement LearningInternational Journal of Intelligent Systems10.1155/2023/88351112023Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1155/2023/8835111
Wang HTang WLuo J(2023)A Network Fault Prediction-Based Service Migration Approach for Unstable Mobile Edge EnvironmentWireless Communications & Mobile Computing10.1155/2023/27530852023Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1155/2023/2753085
Zhang XChang ZWu HLi YChen JTan JLi FCui B(2023)A Unified and Efficient Coordinating Framework for Autonomous DBMS TuningProceedings of the ACM on Management of Data10.1145/35893311:2(1-26)Online publication date: 20-Jun-2023
https://dl.acm.org/doi/10.1145/3589331
Nie XMiao XWang ZYang ZXue JMa LCao GCui B(2023)FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device PlacementProceedings of the ACM on Management of Data10.1145/35889641:1(1-19)Online publication date: 30-May-2023
https://dl.acm.org/doi/10.1145/3588964
Huang SWang ZZhang XTu YLi ZCui B(2023)DBPA: A Benchmark for Transactional Database Performance AnomaliesProceedings of the ACM on Management of Data10.1145/35889261:1(1-26)Online publication date: 30-May-2023
https://dl.acm.org/doi/10.1145/3588926
Show More Cited By

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents