research-article

Maestro: quality-of-service in large disk arrays

Authors:

Pradeep Padala,

Sharad Singhal,

Kang ShinAuthors Info & Claims

ICAC '11: Proceedings of the 8th ACM international conference on Autonomic computing

Pages 245 - 254

https://doi.org/10.1145/1998582.1998638

Published: 14 June 2011 Publication History

Abstract

Provisioning storage in disk arrays is a difficult problem because many applications with different workload characteristics and priorities share resources provided by the array. Currently, storage in disk arrays is statically partitioned, leading to difficult choices between over-provisioning to meet peak demands and resource sharing to meet efficiency targets. In this paper, we present Maestro, a feedback controller that can manage resources on large disk arrays to provide performance differentiation among multiple applications. Maestro monitors the performance of each application and dynamically allocates the array resources so that diverse performance requirements can be met without static partitioning. It supports multiple performance metrics (e.g., latency and throughput) and application priorities so that important applications receive better performance in case of resource contention. By ensuring that high-priority applications sharing storage with other applications obtain the performance levels they require, Maestro makes it possible to use storage resources efficiently. We evaluate Maestro using both synthetic and real-world workloads on a large, commercial disk array. Our experiments indicate that Maestro can reliably adjust the allocation of disk array resources to achieve application performance targets.

References

[1]

T.F. Abdelzaher, K.G. Shin, and N. Bhatti. Performance guarantees for web server end-systems: A control-theoretical approach. IEEE Transactions on Parallel and Distributed Systems, 13, 2002.

Digital Library

[2]

J. Bruno et al. Disk scheduling with quality of service guarantees. In Proc. of the IEEE Int'l Conf. on Multimedia Computing and Systems, Vol. 2, 1999.

Digital Library

[3]

D.D. Chambliss et al. Performance virtualization for large-scale storage systems. In Proc. of Symp. on Reliable Distributed Systems, October 2003.

[4]

A. Demers, S. Keshav, and S. Shenker. Analysis and simulation of a fair queuing algorithm. Journal of Internetworking Research and Experience, 1(1):3--26, September 1990.

[5]

Y. Diao et al. MIMO control of an apache web server: Modeling and controller design. In Proc. of American Control Conference (ACC), 2002.

[6]

A. Gulati, I. Ahmad, and C. Waldspurger. PARDA: Proportional allocation of resources in distributed storage access. In Proc. of USENIX FAST, Feb 2009.

Digital Library

[7]

A. Gulati, A. Merchant, and P. Varman. pClock: An arrival curve based approach for QoS in shared storage systems. In Proc. of ACM SIGMETRICS, pages 13--24, June 2007.

Digital Library

[8]

A. Gulati, A. Merchant, and P.J. Varman. mClock: Handling throughput variability for hypervisor IO scheduling. In Proc. of USENIX OSDI'10, October 2010.

Digital Library

[9]

J. L. Hellerstein. Designing in control engineering of computing systems. In Proc. of American Control Conference, 2004.

[10]

L. Huang, G. Peng, and T-C. Chiueh. Multi-dimensional storage virtualization. In Proc. of ACM SIGMETRICS, pages 14--24, June 2004.

Digital Library

[11]

W. Jin, J.S. Chase, and J. Kaur. Interposed proportional sharing for a storage service utility. In Proc. of ACM SIGMETRICS, pages 37--48, 2004.

Digital Library

[12]

A. Kamra, V. Misra, and E. Nahum. Yaksha: A self-tuning controller for managing the performance of 3-tiered web sites. In Proc. of the International Workshop on Quality of Service (IWQoS), June 2004.

[13]

C. Karamanolis, M. Karlsson, and X. Zhu. Designing controllable computer systems. In Proc. of HOTOS, pages 49--54, June 2005.

Digital Library

[14]

M. Karlsson, C.T. Karamanolis, and X. Zhu. Triage: Performance differentiation for storage systems using adaptive control. ACM Transactions on Storage, 1(4):457--480, 2005.

Digital Library

[15]

X. Liu, X. Zhu, P. Padala, Z. Wang, and S. Singhal. Optimal multivariate control for differentiated services on a shared hosting platform. In Proc. of the IEEE Conference on Decision and Control (CDC), 2007.

[16]

Y. Lu, T.F. Abdelzaher, and A. Saxena. Design, implementation, and evaluation of differentiated caching services. IEEE Transactions on Parallel and Distributed Systems, 15(5), May 2004.

Digital Library

[17]

C.R. Lumb, A. Merchant, and G.A. Alvarez. Fa\c cade: Virtual storage devices with performance guarantees. In Proc. of USENIX FAST, 2003.

Digital Library

[18]

M.P. Mesnier et al. Modeling the relative fitness of storage. In ACM SIGMETRICS, pages 37--48, 2007.

Digital Library

[19]

R. Nathuji, A. Kansal, and A. Ghaffarkhah. Q-Clouds: Managing performance interference effects for QoS-aware clouds. In Proc. of ACM EuroSys, April 2010.

Digital Library

[20]

P. Padala et al. Adaptive control of virutalized resources in utility computing environments. In Proc. of ACM EuroSys, March 2007.

Digital Library

[21]

P. Padala et al. Automated control of multiple virtualized resources. In Proc. of ACM EuroSys, March 2009.

Digital Library

[22]

A. Povzner et al. Efficient guaranteed disk request scheduling with Fahrrad. SIGOPS Oper. Syst. Rev., 42(4):13--25, 2008.

Digital Library

[23]

E. Varki et al. Issues and challenges in the performance analysis of real disk arrays. IEEE Trans. Parallel Distrib. Syst., 15(6):559--574, 2004.

Digital Library

[24]

M. Wachs, M. Abd-El-Malek, E. Thereska, and G.R. Ganger. Argon: performance insulation for shared storage servers. In Proc. of USENIX FAST, 2007.

Digital Library

[25]

Y. Wang and A. Merchant. Proportional-share scheduling for distributed storage systems. In Proc. of USENIX FAST, Feb 2007.

Digital Library

[26]

T.M. Wong, R.A. Golding, C. Lin, and R.A. Becker-Szendy. Zygaria: Storage performance as managed resource. In Proc. of RTAS, Apr 2006.

Digital Library

[27]

J.C. Wu and S.A. Brandt. The design and implementation of AQuA: an adaptive quality of service aware object-based storage device. In Proc. of IEEE/NASA MSST, May 2006.

[28]

J. Zhang et al. Storage performance virtualization via throughput and latency control. In Proc. of MASCOTS, September 2005.

Digital Library

[29]

Y. Zhang et al. Friendly virtual machines: leveraging a feedback-control model for application adaptation. In Proc. of the ACM Virtual Execution Environments Conference (VEE), 2005.\endthebibliography

Digital Library

Cited By

Habibi F(2024)PhD Forum: Towards Metastable-Failure-Free Distributed Transaction Systems2024 43rd International Symposium on Reliable Distributed Systems (SRDS)10.1109/SRDS64841.2024.00038(318-321)Online publication date: 30-Sep-2024
https://doi.org/10.1109/SRDS64841.2024.00038
Habibi FLorido-Botran TShowail ASturman DNawab F(2024)MSF-Model: Queuing-Based Analysis and Prediction of Metastable Failures in Replicated Storage Systems2024 43rd International Symposium on Reliable Distributed Systems (SRDS)10.1109/SRDS64841.2024.00013(12-22)Online publication date: 30-Sep-2024
https://doi.org/10.1109/SRDS64841.2024.00013
Bronson NAghayev ACharapko AZhu TAngel SKasikci BKohler E(2021)Metastable failures in distributed systemsProceedings of the Workshop on Hot Topics in Operating Systems10.1145/3458336.3465286(221-227)Online publication date: 1-Jun-2021
https://dl.acm.org/doi/10.1145/3458336.3465286
Show More Cited By

Index Terms

Maestro: quality-of-service in large disk arrays

Recommendations

HPDA: A hybrid parity-based disk array for enhanced performance and reliability

Flash-based Solid State Drive (SSD) has been productively shipped and deployed in large scale storage systems. However, a single flash-based SSD cannot satisfy the capacity, performance and reliability requirements of the modern storage systems that ...
A Performance Evaluation of RAID Architectures

In today's computer systems, the disk I/O subsystem is often identified as the major bottleneck to system performance. One proposed solution is the so-called redundant array of inexpensive disks (RAID). In this paper, we examine the performance of two ...
Parity logging disk arrays

Parity-encoded redundant disk arrays provide highly reliable, cost-effective secondary storage with high performance for reads and large writes. Their performance on small writes, however, is much worse than mirrored disks—the traditional, highly ...

Comments

Information & Contributors

Information

Published In

ICAC '11: Proceedings of the 8th ACM international conference on Autonomic computing

June 2011

278 pages

ISBN:9781450306072

DOI:10.1145/1998582

General Chairs:
Hartmut Schmeck
Karlsruhe Institute of Technology, Germany
,
Wolfgang Rosenstiel
University of Tübingen, Germany
,
Program Chairs:
Tarek Abdelzaher
University of Illinois at Urbana-Champaign, USA
,
Joseph Hellerstein
Google, USA

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

IEEE
University of Arizona: University of Arizona
SIGARCH: ACM Special Interest Group on Computer Architecture

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 June 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICAC '11

Sponsor:

University of Arizona
SIGARCH

ICAC '11: 8th International Conference on Autonomic Computing

June 14 - 18, 2011

Karlsruhe, Germany

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

30
Total Citations
View Citations
223
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 24 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Habibi F(2024)PhD Forum: Towards Metastable-Failure-Free Distributed Transaction Systems2024 43rd International Symposium on Reliable Distributed Systems (SRDS)10.1109/SRDS64841.2024.00038(318-321)Online publication date: 30-Sep-2024
https://doi.org/10.1109/SRDS64841.2024.00038
Habibi FLorido-Botran TShowail ASturman DNawab F(2024)MSF-Model: Queuing-Based Analysis and Prediction of Metastable Failures in Replicated Storage Systems2024 43rd International Symposium on Reliable Distributed Systems (SRDS)10.1109/SRDS64841.2024.00013(12-22)Online publication date: 30-Sep-2024
https://doi.org/10.1109/SRDS64841.2024.00013
Bronson NAghayev ACharapko AZhu TAngel SKasikci BKohler E(2021)Metastable failures in distributed systemsProceedings of the Workshop on Hot Topics in Operating Systems10.1145/3458336.3465286(221-227)Online publication date: 1-Jun-2021
https://dl.acm.org/doi/10.1145/3458336.3465286
Patel TGarg RTiwari DNoh SWelch B(2020)GIFTProceedings of the 18th USENIX Conference on File and Storage Technologies10.5555/3386691.3386702(103-120)Online publication date: 24-Feb-2020
https://dl.acm.org/doi/10.5555/3386691.3386702
Peng YVarman PYadgar GNoh S(2019)Fair-EDFProceedings of the 11th USENIX Conference on Hot Topics in Storage and File Systems10.5555/3357062.3357070(6-6)Online publication date: 8-Jul-2019
https://dl.acm.org/doi/10.5555/3357062.3357070
Peng YLiu QVarman P(2019)Latency Fairness Scheduling for Shared Storage Systems2019 IEEE International Conference on Networking, Architecture and Storage (NAS)10.1109/NAS.2019.8834732(1-8)Online publication date: Aug-2019
https://doi.org/10.1109/NAS.2019.8834732
Nagao TTanabe NYokoyama KTaniguchi H(2019)Evaluation of I/O Performance Regulating Function with a Virtual MachineAdvances in Networked-based Information Systems10.1007/978-3-030-29029-0_63(641-649)Online publication date: 15-Aug-2019
https://doi.org/10.1007/978-3-030-29029-0_63
Gugnani SLu XPanda D(2018)Analyzing, Modeling, and Provisioning QoS for NVMe SSDs2018 IEEE/ACM 11th International Conference on Utility and Cloud Computing (UCC)10.1109/UCC.2018.00033(247-256)Online publication date: Dec-2018
https://doi.org/10.1109/UCC.2018.00033
Klimovic ALitz HKozyrakis C(2017)ReFlexACM SIGARCH Computer Architecture News10.1145/3093337.303773245:1(345-359)Online publication date: 4-Apr-2017
https://dl.acm.org/doi/10.1145/3093337.3037732
Klimovic ALitz HKozyrakis C(2017)ReFlexACM SIGPLAN Notices10.1145/3093336.303773252:4(345-359)Online publication date: 4-Apr-2017
https://dl.acm.org/doi/10.1145/3093336.3037732
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents