research-article

Fitting Statistical Models of Random Search in Simulation Studies

Author:

Russell C. H. ChengAuthors Info & Claims

ACM Transactions on Modeling and Computer Simulation (TOMACS), Volume 23, Issue 3

Article No.: 15, Pages 1 - 22

https://doi.org/10.1145/2499913.2499914

Published: 01 July 2013 Publication History

Abstract

We consider optimization of expected system performance by random search. There are two sources of random variation in this process: (i) a search-induced variability because the expected performance of the system will vary randomly according to the alternatives randomly selected for examination, and (ii) a simulation induced variability, because there will be random error in estimating expected system performance from finite simulation runs. We show that, in altering the balance between these two sources of variability, three distinct forms of asymptotic behavior of the estimate of the optimal expected system performance are possible. The form of the asymptotic results shows that they may be not be easy to apply in practical work. As an alternative, a methodology for fitting a statistical model that accounts for both types of variability is suggested. This then allows the distributional properties of quantities of interest, like the optimum performance value and the best value obtained by the search, to be estimated by resampling and which also allows a test of goodness of fit of the model. Four numerical examples are given.

Supplementary Material

PDF File (a15-cheng_appendix.pdf)

The proof is given in an electronic appendix, available online in the ACM Digital Library.

Download
79.59 KB

References

[1]

Anderson, T. W. and Darling, D. 1952. Asymptotic theory of certain ‘goodness of fit’ criteria based on stochastic processes. Ann. Math. Stat. 23, 193--212.

[2]

Archetti, F., Betrò, B., and Steffè, S. 1977. A theoretical framework for global optimization via random sampling. Tech. rep., Cuaderni del Dipartimento di Ricerca Operative e Scienze Statische, Università di Pisa.

[3]

Berman, S. M. 1992. The tail of the convolution of densities and its application to a model of HIV-latency time. Ann. Appl. Probab. 2, 481--502.

[4]

Bickel, P. J. and Freedman, D. A. 1981 Some asymptotic theory for the bootstrap. Ann. Stat. 9, 1196--1217.

[5]

Carroll, R. J. and Hall, P. 1988. Optimal rates of convergence for deconvolving a density. J. Amer. Statist. Assoc. 83, 1184--1186.

[6]

Cheng, R. C. H. 2006. Validating and comparing simulation models using resampling. J. Simulation. 1, 53--63.

[7]

Cheng, R. C. H. 2007. DES as a real time decision making tool. An application to fire service emergency cover. In Proceedings of the INFORMS Simulation Society Workshop. C.-H. Chen and S. G. Henderson Eds., http://www.informs.org/Community/Simulation-Society/Other-Conferences-and-Workshops/Society-Research-Workshop2/2007-Simulation-Workshop, pdf #17.

[8]

Cheng, R. C. H. and Amin, N. A. K. 1983. Estimating parameters in continuous univariate distributions with a shifted origin. J. Roy. Stat. Soc. Ser. B, 45, 394--403.

[9]

Chia, Y. L. 2005 Simulation-based estimation with application in biomedical research. Ph.D. thesis, Stanford University.

[10]

Chia, Y. L. and Glynn, P. W. 2007. Optimal convergence rate for random search. In Proceedings of the INFORMS Simulation Society Workshop. C.-H. Chen and S. G. Henderson Eds., http://www.informs.org/Community/Simulation-Society/Other-Conferences-and-Workshops/Society-Research-Workshop2/2007-Simulation-Workshop, pdf #11.

[11]

Davison, A. C. and Hinkley, D. V. 1997. Bootstrap Methods and Their Application. Cambridge University Press, Cambridge, UK.

[12]

de Haan, L. 1981. Estimation of the minimum of a function using order statistics. J. Amer. Statis. Assoc. 76, 467--469.

[13]

Gnedenko, B. V. 1943. Sur la distribution limite du terme maximum d’une serie aleatoire. Ann. Math. 44, 423--453. {English translation by N. L. Johnson in Breakthroughs in Statistics Vol. I. S. Kotz and N. L. Johnson Eds., Springer-Verlag, Berlin, Germany, 195--225.}

[14]

Goldenshluger, A. and Tsybakov, A. 2004. Estimating the endpoint of a distribution in the presence of additive observational errors. Stat. Probab. Lett. 68, 39--49.

[15]

Hjorth, J. S. U. 1994. Computer Intensive Statistical Methods. Chapman and Hall, London.

[16]

Pickands, J. 1975. Statistical inference using extreme order statistics. Ann. Stat. 3, 119--131.

[17]

Ranneby, B. 1984. The maximum spacing method: An estimation method related to the maximum likelihood method. Scand. J. Statist. 11, 93--112.

[18]

Seneta, E. 1976. Regularly varying functions. Lecture Notes in Mathematics, vol. 508. Springer-Verlag, Berlin.

[19]

Smith, R. L. 1987. Estimating tails of probability distributions. Ann. Stat. 15, 1174--1207.

[20]

Wald, A. 1949. A note on the consistency of the maximum likelihood estimate. Ann. Math. Stat. 20, 595--601.

[21]

Yakowitz, S., L’Ecuyer, P., and Vásquez-Abad, F. 2000. Global stochastic optimization with low-dispersion point sets. Oper. Res. 48, 939--950.

Digital Library

Cited By

Cheng R(2015)Supporting Time-Critical Decision Making with Real Time SimulationsUncertainty Management in Simulation-Optimization of Complex Systems10.1007/978-1-4899-7547-8_1(3-26)Online publication date: 2015
https://doi.org/10.1007/978-1-4899-7547-8_1
Andradóttir S(2014)A Review of Random Search MethodsHandbook of Simulation Optimization10.1007/978-1-4939-1384-8_10(277-292)Online publication date: 18-Sep-2014
https://doi.org/10.1007/978-1-4939-1384-8_10
Cheng RHill RKuhl M(2013)Supporting time-critical decision making with real time simulationsProceedings of the 2013 Winter Simulation Conference: Simulation: Making Decisions in a Complex World10.5555/2675983.2676010(229-240)Online publication date: 8-Dec-2013
https://dl.acm.org/doi/10.5555/2675983.2676010
Show More Cited By

Index Terms

Fitting Statistical Models of Random Search in Simulation Studies

Recommendations

Assessing statistical reliability of LiNGAM via multiscale bootstrap
ICANN'10: Proceedings of the 20th international conference on Artificial neural networks: Part III

Structural equation models have been widely used to study causal relationships between continuous variables. Recently, a non-Gaussian method called LiNGAM was proposed to discover such causal models and has been extended in various directions. An ...
Analysis of survival data having time-dependent covariates

Cox's proportional hazards model has been widely used for the analysis of treatment and prognostic effects with censored survival data. In this paper, we propose a neural network model based on bootstrapping to estimate the survival function and predict ...
Fitting semiparametric cure models

Survival data with a sizable cure fraction are commonly encountered in some cancer clinical researches and the semiparametric proportional hazards cure model has been recently investigated to analyze such data. However the estimation method of the model ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Modeling and Computer Simulation

ACM Transactions on Modeling and Computer Simulation Volume 23, Issue 3

July 2013

126 pages

ISSN:1049-3301

EISSN:1558-1195

DOI:10.1145/2499913

Editor-in-chief:
Adelinde M. Uhrmacher

Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2013

Accepted: 01 October 2011

Revised: 01 October 2011

Received: 01 February 2008

Published in TOMACS Volume 23, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
191
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cheng R(2015)Supporting Time-Critical Decision Making with Real Time SimulationsUncertainty Management in Simulation-Optimization of Complex Systems10.1007/978-1-4899-7547-8_1(3-26)Online publication date: 2015
https://doi.org/10.1007/978-1-4899-7547-8_1
Andradóttir S(2014)A Review of Random Search MethodsHandbook of Simulation Optimization10.1007/978-1-4939-1384-8_10(277-292)Online publication date: 18-Sep-2014
https://doi.org/10.1007/978-1-4939-1384-8_10
Cheng RHill RKuhl M(2013)Supporting time-critical decision making with real time simulationsProceedings of the 2013 Winter Simulation Conference: Simulation: Making Decisions in a Complex World10.5555/2675983.2676010(229-240)Online publication date: 8-Dec-2013
https://dl.acm.org/doi/10.5555/2675983.2676010
Cheng R(2013)Supporting time-critical decision making with real time simulations2013 Winter Simulations Conference (WSC)10.1109/WSC.2013.6721422(229-240)Online publication date: Dec-2013
https://doi.org/10.1109/WSC.2013.6721422

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents