article

Characterizing spot price dynamics in public cloud environments

Authors:

Ruppa K. Thulasiram,

Rajkumar BuyyaAuthors Info & Claims

Future Generation Computer Systems, Volume 29, Issue 4

Pages 988 - 999

https://doi.org/10.1016/j.future.2012.06.012

Published: 01 June 2013 Publication History

Abstract

The surge in demand for utilizing public Cloud resources has introduced many trade-offs between price, performance and recently reliability. Amazon's Spot Instances (SIs) create a competitive bidding option for public Cloud users at lower prices without providing reliability on services. It is generally believed that SIs reduce monetary cost to the Cloud users, however it appears from the literature that their characteristics have not been explored and reported. We believe that characterization of SIs is fundamental in the design of stochastic scheduling algorithms and fault tolerant mechanisms in public Cloud environments for the spot market. In this paper, we have done a comprehensive analysis of SIs based on one year price history in four data centers of Amazon's EC2. For this purpose, we have analyzed all different types of SIs in terms of spot price and the inter-price time (time between price changes) and determined the time dynamics for spot price in hour-in-day and day-of-week. Moreover, we have proposed a statistical model that fits well these two data series. The results reveal that we are able to model spot price dynamics as well as the inter-price time of each SI by a mixture of Gaussians distribution with three or four components. The proposed model is validated through extensive simulations, which demonstrate that our model exhibits a good degree of accuracy under realistic working conditions.

References

[1]

Varia, J., . In: Best Practices in Architecting Cloud Applications in the AWS Cloud, Wiley Press. pp. 459-490.

[2]

Amazon Inc., Amazon Elastic Compute Cloud, Amazon EC2. http://aws.amazon.com/ec2.

[3]

S. Yi, D. Kondo, A. Andrzejak, Reducing costs of Spot instances via checkpointing in the Amazon elastic compute cloud, in: 3rd IEEE International Conference on Cloud Computing, 2010, pp. 236-243.

Digital Library

[4]

Yi, S., Kondo, D. and Andrzejak, A., Monetary cost-aware checkpointing and migration on Amazon Cloud Spot instances. IEEE Transactions on Services Computing. 236-243.

[5]

M. Mattess, C. Vecchiola, R. Buyya, Managing peak loads by leasing cloud infrastructure services from a spot market, in: 12th IEEE International Conference on High Performance Computing and Communications, 2010, pp.¿180-188.

Digital Library

[6]

Ortuno, F.M. and Harder, U., A stochastic calculus model for the spot price of computing power. In: The UK Performance Engineering Workshop, UKPEW.

[7]

Vanmechelen, K., Depoorter, W. and Broeckhove, J., Combining futures and spot markets: a hybrid market approach to economic grid resource management. Journal of Grid Computing. v9. 81-94.

[8]

http://cloudexchange.org/

[9]

Amazon Inc., Amazon Discussion Forums. https://forums.aws.amazon.com.

[10]

B. Javadi, R. Buyya, Comprehensive statistical analysis and modeling of Spot instances in public Cloud environments, Research Report CLOUDS-TR-2011-1, Cloud Computing and Distributed Systems Laboratory. The University of Melbourne, March 2011.

[11]

S. Wee, Debunking real-time pricing in Cloud computing, in: 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid, 2011, pp. 585-590.

Digital Library

[12]

O.A. Ben-Yehuda, M. Ben-Yehuda, A. Schuster, D. Tsafrir, Deconstructing Amazon EC2 Spot instance pricing, in: 3rd IEEE International Conference on Cloud Computing Technology and Science, CloudCom, 2011, pp. 304-311.

[13]

S. Chaisiri, R. Kaewpuang, B.-S. Lee, D. Niyato, Cost minimization for provisioning virtual servers in Amazon elastic compute cloud, in: 19th IEEE International Symposium on Modeling, Analysis Simulation of Computer and Telecommunication Systems, MASCOTS, 2011, pp. 85-95.

Digital Library

[14]

W. Voorsluys, S.K. Garg, R. Buyya, Provisioning spot market Cloud resources to create cost-effective virtual clusters, in: 11th International Conference Algorithms and Architectures for Parallel Processing, ICA3PP, 2011, pp.¿395-408.

Digital Library

[15]

A. Andrzejak, D. Kondo, S. Yi, Decision model for Cloud computing under SLA constraints, in: 18th IEEE/ACM International Symposium on Modelling, Analysis and Simulation of Computer and Telecommunication Systems, MASCOTS, 2010, pp. 257-266.

[16]

M. Mazzucco, M. Dumas, Achieving performance and availability guarantees with Spot instances, in: 13th IEEE International Conference on High Performance Computing and Communications, HPCC, 2011, pp. 296-303.

Digital Library

[17]

N. Chohan, C. Castillo, M. Spreitzer, M. Steinder, A. Tantawi, C. Krintz, See spot run: using Spot instances for MapReduce workflows, in: the 2nd USENIX Conference on Hot Topics in Cloud Computing, HotCloud'10, 2010, pp. 7-13.

[18]

Zhang, Q., Gurses, E., Boutaba, R. and Xiao, J., Dynamic resource allocation for spot markets in clouds. In: 11th USENIX Conference on Hot Topics in Management of Internet, Cloud, and Enterprise Networks and Services, Hot-ICE'11, USENIX Association, Berkeley, CA, USA. pp. 1-6.

[19]

Q. Zhang, Q. Zhu, R. Boutaba, Dynamic resource allocation for spot markets in Cloud computing environments, in: 4th IEEE International Conference on Utility and Cloud Computing, UCC, 2011, pp. 178-185.

Digital Library

[20]

M.R. Rahman, Y. Lu, I. Gupta, Risk aware resource allocation for clouds, Technical Report 2011-07-11, University of Illinois at Urbana-Champaign, July 2011.

[21]

D.G. Feitelson, Workload Modeling for Computer Systems Performance Evaluation, e-Book, 2011. http://www.cs.huji.ac.il/~feit/wlmod/.

[22]

Li, H., Realistic workload modeling and its performance impacts in large-scale escience grids. IEEE Transactions on Parallel and Distributed Systems. v21 i4. 480-493.

[23]

D. Kondo, B. Javadi, A. Iosup, D.H.J. Epema, The Failure Trace Archive: enabling comparative analysis of failures in diverse distributed systems, in: 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing, CCGRID, 2010, pp. 398-407.

[24]

D. Kondo, A. Andrzejak, D.P. Anderson, On correlated availability in internet distributed systems, in: 9th IEEE/ACM International Conference on Grid Computing, Grid 2008, 2008, pp. 276-283.

[25]

Li, H., Muskulus, M. and Wolters, L., Modeling correlated workloads by combining model based clustering and a localized sampling algorithm. In: Proceedings of the 21st Annual International Conference on Supercomputing, ICS'07, ACM, New York, NY, USA. pp. 64-72.

[26]

Fraley, C. and Raftery, A.E., Model-based clustering, discriminant analysis, and density estimation. Journal of the American Statistical Association. v97 i458. 611-631.

[27]

B. Javadi, D. Kondo, J.-M. Vincent, D.P. Anderson, Mining for statistical availability models in large-scale distributed systems: an empirical study of SETI@home, in: 17th IEEE/ACM International Symposium on Modelling, Analysis and Simulation of Computer and Telecommunication Systems, MASCOTS, 2009, pp. 1-10.

[28]

Calheiros, R.N., Ranjan, R., Beloglazov, A., De Rose, C.A.F. and Buyya, R., CloudSim: a toolkit for modeling and simulation of Cloud computing environments and evaluation of resource provisioning algorithms. Software: Practice and Experience. v41 i1. 23-50.

[29]

Iosup, A., Li, H., Jan, M., Anoep, S., Dumitrescu, C., Wolters, L. and Epema, D.H.J., The grid workloads archive. Future Generation Computer Systems. v24 i7. 672-686.

[30]

Vecchiola, C., Calheiros, R.N., Karunamoorthy, D. and Buyya, R., Deadline-driven provisioning of resources for scientific applications in hybrid clouds with Aneka. Future Generation Computer Systems. v28 i1. 58-65.

Cited By

Kim KLee KChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Making Cloud Spot Instance Interruption Events VisibleProceedings of the ACM Web Conference 202410.1145/3589334.3645548(2998-3009)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645548
Singh VShivendu SDutta K(2022)Spot instance similarity and substitution effect in cloud spot marketDecision Support Systems10.1016/j.dss.2022.113815159:COnline publication date: 1-Aug-2022
https://dl.acm.org/doi/10.1016/j.dss.2022.113815
Ivashko EIvashko ASafonov GTchernykh A(2020)Cost-Efficient Strategy in Clouds with Spot Price UncertaintyAutomation and Remote Control10.1134/S000511792004013X81:4(731-745)Online publication date: 1-Apr-2020
https://dl.acm.org/doi/10.1134/S000511792004013X
Show More Cited By

Recommendations

How Small and Medium Enterprises SMEs Should Bid for Spot Instances of Amazon's EC2 Cloud

In cloud service provisioning, spot instances are spare slots for which it has no pre-booking, unlike reserved or on-demand instances for which a cloud service provider CSP has a priori booking. CSPs like Amazon prefer spot instance approach to sell ...
Statistical Modeling of Spot Instance Prices in Public Cloud Environments
UCC '11: Proceedings of the 2011 Fourth IEEE International Conference on Utility and Cloud Computing

The surge in demand for utilizing public Cloud resources has introduced many trade-offs between price, performance and recently reliability. Amazon's Spot Instances (SIs) create a competitive bidding option for the public Cloud users at lower prices ...
Price forecasting for spot instances in Cloud computing

Big data applications usually need to rent a large number of virtual machines from Cloud computing providers. As a result of the policies employed by Cloud providers, the prices of spot virtual machine instances behavior stochastically. Spot prices (...

Comments

Information & Contributors

Information

Published In

cover image Future Generation Computer Systems

Future Generation Computer Systems Volume 29, Issue 4

June, 2013

186 pages

ISSN:0167-739X

Issue’s Table of Contents

Copyright © Elsevier B.V. © 2012.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 01 June 2013

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

23
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kim KLee KChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Making Cloud Spot Instance Interruption Events VisibleProceedings of the ACM Web Conference 202410.1145/3589334.3645548(2998-3009)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645548
Singh VShivendu SDutta K(2022)Spot instance similarity and substitution effect in cloud spot marketDecision Support Systems10.1016/j.dss.2022.113815159:COnline publication date: 1-Aug-2022
https://dl.acm.org/doi/10.1016/j.dss.2022.113815
Ivashko EIvashko ASafonov GTchernykh A(2020)Cost-Efficient Strategy in Clouds with Spot Price UncertaintyAutomation and Remote Control10.1134/S000511792004013X81:4(731-745)Online publication date: 1-Apr-2020
https://dl.acm.org/doi/10.1134/S000511792004013X
Zhu ZPeng JLiu KZhang X(2020)A game-based resource pricing and allocation mechanism for profit maximization in cloud computingSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-019-04183-024:6(4191-4203)Online publication date: 1-Mar-2020
https://dl.acm.org/doi/10.1007/s00500-019-04183-0
Baughman MCaton SHaas CChard RWolski RFoster IChard KNicolae BCostan ADuplyakin DGainaru AUta ACosta C(2019)Deconstructing the 2017 Changes to AWS Spot Market PricingProceedings of the 10th Workshop on Scientific Cloud Computing10.1145/3322795.3331465(19-26)Online publication date: 17-Jun-2019
https://dl.acm.org/doi/10.1145/3322795.3331465
Araujo Neto JPianto DRalha C(2019)MULTSJournal of Systems Architecture: the EUROMICRO Journal10.1016/j.sysarc.2019.101651101:COnline publication date: 1-Dec-2019
https://dl.acm.org/doi/10.1016/j.sysarc.2019.101651
Andreadis GVersluis LMastenbroek FIosup A(2018)A reference architecture for datacenter schedulingProceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis10.5555/3291656.3291706(1-15)Online publication date: 11-Nov-2018
https://dl.acm.org/doi/10.5555/3291656.3291706
(2018)Virtual resource pricing scheme in cloud platformsInternational Journal of Networking and Virtual Organisations10.1504/IJNVO.2018.09067618:1(67-79)Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.1504/IJNVO.2018.090676
Baughman MHaas CWolski RFoster IChard K(2018)Predicting Amazon Spot Prices with LSTM NetworksProceedings of the 9th Workshop on Scientific Cloud Computing10.1145/3217880.3217881(1-7)Online publication date: 11-Jun-2018
https://dl.acm.org/doi/10.1145/3217880.3217881
Andreadis GVersluis LMastenbroek FIosup A(2018)A reference architecture for datacenter schedulingProceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis10.1109/SC.2018.00040(1-15)Online publication date: 11-Nov-2018
https://dl.acm.org/doi/10.1109/SC.2018.00040
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents