research-article

Simulation optimization using metamodels

Author:

Russell R. BartonAuthors Info & Claims

WSC '09: Winter Simulation Conference

Pages 230 - 238

Published: 13 December 2009 Publication History

Abstract

Many iterative optimization methods are designed to be used in conjunction with deterministic objective functions. These optimization methods can be difficult to apply to an objective generated by a discrete-event simulation, due to the stochastic nature of the response(s) and the potentially extensive run times. A metamodel aids simulation optimization by providing a deterministic objective with run times that are generally much shorter than the original discrete-event simulation. Polynomial metamodels generally provide only local approximations, and so a series of metamodels must be fit as the optimization progresses. Other classes of metamodels can provide global fit; fitting can be done either by constructing the global model once at the start of the optimization, or by using the optimization results to identify additional discrete-event runs to refine the global model. This tutorial surveys both local and global metamodel-based optimization methods.

References

[1]

Alexandrov, N. M., J. E. Dennis, Jr., R. M. Lewis and V. Torczon. 1998. A trust region framework for managing the use of approximation models in optimization. Structural Optimization 15(1): 16--23.

[2]

Allen, T. T., M. A. Bernshteyn and K. Kabiri-Bamoradian. 2003. Constructing meta-models for computer experiments. Journal of Quality Technology 35(3): 264--274.

[3]

Ankenman, B., B. L. Nelson and J. Staum. 2008. Stochastic kriging for simulation metamodeling. In Proceedings of the 2008 Winter Simulation Conference, ed. S. J. Mason, R. R. Hill, L. Mönch, O. Rose, T. Jefferson, J. W. Fowler, 362--370. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[4]

April, J. F. Glover, J. P. Kelly and M. Laguna. 2003. Simulation-based optimization: practical introduction to simulation optimization. In Proceedings of the 2003 Winter Simulation Conference, ed. S. Chick, P. J. Sánchez, D. Ferrin, and D. J. Morrice, 71--78. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[5]

Barton, R. R. 1992. Metamodels for simulation input-output relations. In Proceedings of the 1992 Winter Simulation Conference, ed. J. J. Swain, D. Goldsman, R. C. Crain, and J. R. Wilson, 289--299. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[6]

Barton, R. R. 1998. Simulation metamodels. In Proceedings of the 1998 Winter Simulation Conference, ed. D. J. Medeiros, E. F. Watson, J. S. Carson and M. S. Manivannan, 167--174. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[7]

Barton, R. R. and M. Meckesheimer. 2006. Metamodel-based simulation optimization. In Simulation. Handbooks in Operations Research and Management Science vol. 13. Amsterdam: Elsevier B. V.

[8]

van Beers, W. C. M. and J. P. C. Kleijnen. 2008. Customized sequential designs for random simulation experiments: kriging metamodeling and bootstrapping. European Journal of Operational Research 186(3): 1099--1113.

[9]

Bernardo, M. C., R. Buck, L. Liu, W. A. Nazaret, J. Sacks and W. J. Welch. 1992. Integrated circuit design optimization using a sequential strategy. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 11(3): 361--372.

Digital Library

[10]

Bettonvil, B., E. del Castillo and J. P. C. Kleijnen. 2009. Statistical testing of optimality conditions in multiresponse simulation-based optimization. European Journal of Operational Research 199(2): 448--458.

[11]

Biles, W. E. 1974. A gradient regression search procedure for simulation experimentation. In Proceedings of the 1974 Winter Simulation Conference, ed. H. Highland, H. Steinberg and M. F. Morris, 491--497. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[12]

Biles, W. E., J. P. C. Kleijnen, W. C. van Beers and I. Nieuwenhuyse. 2007. Kriging metamodeling in constrained simulation optimization: an explorative study. In Proceedings of the 2007 Winter Simulation Conference, ed. S. G. Henderson, B. Biller, M.-H. Hsieh, J. Shortle, J. D. Tew, and R. R. Barton, 355--362. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[13]

Booker, A. J., J. E. Dennis, Jr., P. D. Frank, D. B. Serafini, V. Torczon and M. W. Trosset. 1999. A rigorous framework for optimization of expensive functions by surrogates. Structural and Multidisciplinary Optimization 17(1): 1--13.

[14]

Box, G. E. P. and K. B. Wilson. 1951. On the experimental attainment of optimum conditions. Journal of the Royal Statistical Society. Series B (Methodological) 13: 1--45.

[15]

del Castillo, E. D. 2007. Process optimization. New York: Springer.

[16]

Chan, W. K. and L. W. Schruben. 2006. Response gradient estimation using mathematical programming models of discrete-event system sample paths. In Proceedings of the 2006 Winter Simulation Conference, ed. L. F. Perrone, F. P. Wieland, J. Liu, B. G. Lawson, D. M. Nicol, and R. M. Fujimoto, 272--278. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[17]

Chang, K., L. J. Hong and H. Wan. 2007. Stochastic trust region gradient-free method (strong): a new response-surface-based algorithm in simulation optimization. In Proceedings of the 2007 Winter Simulation Conference, ed. S. G. Henderson, B. Biller, M.-H. Hsieh, J. Shortle, J. D. Tew, and R. R. Barton, 346--354. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[18]

Cressie, N. A. C. 1993. Statistics for spatial data. New York: John Wiley and Sons.

[19]

Currin, C., T. Mitchell, M. Morris and D. Ylvisaker. 1991. Bayesian prediction of deterministic functions, with applications to the design and analysis of computer experiments. Journal of the American Statistical Association 86(416): 953--963.

[20]

Fu, M., F. Glover and J. April. 2005. Simulation optimization: a review, new developments, and applications. In Proceedings of the 2005 Winter Simulation Conference, ed. M. E. Kuhl, N. M. Steiger, F. B. Armstrong, and J. A. Joines, 83--95. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[21]

Handcock, M. S. and M. L. Stein. 1993. A Bayesian analysis of kriging. Technometrics 35(4): 403--410.

[22]

Huang, D., T. T. Allen, W. Notz and N. Zeng. 2006. Global optimization of stochastic black-box systems via sequential krig-ing meta-models. Journal of Global Optimization 34: 441--466.

Digital Library

[23]

Jones, D., M. Schonlau and W. Welch. 1998. Efficient global optimization of expensive black-box functions. Journal of Global Optimization 13: 455--492.

Digital Library

[24]

Kleijnen, J. P. C. 1975. A comment on Blanning's "metamodel for sensitivity analysis: the regression metamodel in simulation." Interfaces 5(3): 21--23.

[25]

Kleijnen, J. P. C. 2007. Regression models and experimental designs: a tutorial for simulation analysts. In Proceedings of the 2007 Winter Simulation Conference, ed. S. G. Henderson, B. Biller, M.-H. Hsieh, J. Shortle, J. D. Tew, and R. R. Barton, 183--194. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[26]

Kleijnen, J. P. C. 2008a. Design and analysis of simulation experiments. New York: Springer.

Digital Library

[27]

Kleijnen, J. P. C. 2008b. Design of experiments: overview. In Proceedings of the 2008 Winter Simulation Conference, ed. S. J. Mason, R. R. Hill, L. Mönch, O. Rose, T. Jefferson, J. W. Fowler, 479--488. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[28]

Kleijnen, J. P. C. 2008c. Response surface methodology for constrained simulation optimization: an overview. Simulation Modelling Practice and Theory 16: 50--64.

[29]

Kleijnen, J. P. C., W. C. M. van Beers and I. Nieuwenhuyse. 2009. Constrained optimization in simulation: a novel approach. European Journal of Operational Research (to appear).

[30]

Kleijnen, J. P. C., D. den Hertog and E. Angun. 2004. Response surface methodology's steepest ascent and step size revisited. European Journal of Operational Research 159(1): 121--131.

[31]

Kleijnen, J. P. C. and R. Y. Rubinstein. 1996. Optimization and sensitivity analysis of computer simulation models by the score function method. European Journal of Operational Research 88: 413--427.

[32]

Krige, D. G. 1951. A statistical approach to some basic mine valuation problems on the Witwatersrand. Journal of the Chemical, Metallurgical and Mining Society of South Africa 52(6): 119--139.

[33]

L'Ecuyer, P. and P. W. Glynn. 1994. Stochastic optimization by simulation: convergence proofs for the GI/G/1 queue in steady-state. Management Science 40(11): 1562--1578.

Digital Library

[34]

Matheron, G. 1962. Traité de Géostatistique Appliquée, Tome 1, Paris: Technip.

[35]

Matheron, G. 1963. Traité de Géostatistique Appliquée, Tome 2, Paris: Technip.

[36]

Morris, M. D., T. J. Mitchell and D. Ylvisaker. 1993. Bayesian design and analysis of computer experiments: use of derivatives in surface prediction. Technometrics 35(3): 243--255.

[37]

Myers, R. H., D. C. Montgomery and C. M. Anderson-Cook. 2009. Response surface methodology. New York: John Wiley and Sons.

[38]

Neddermeijer, H. G., G. J. van Oortmarssen, N. Piersma and R. Dekker. 2000. A framework for response surface methodology for simulation optimization. In Proceedings of the 2000 Winter Simulation Conference, ed. J. A. Joines, R. R. Barton, K. Kang, and P. A. Fishwick, 129--136. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[39]

Nicolai, R. P., R. Dekker, N. Piersma and G. J. van Oortmarssen. 2004. Automated response surface methodology for stochastic optimization models with unknown variance. In Proceedings of the 2004 Winter Simulation Conference, ed. R. G. Ingalls, M. D. Rossetti, J. S. Smith, and B. A. Peters, 491--499. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[40]

Oon, S. J. and L. H. Lee. 2006. The impact of ordinal on response surface methodology. In Proceedings of the 2006 Winter Simulation Conference, ed. L. F. Perrone, F. P. Wieland, J. Liu, B. G. Lawson, D. M. Nicol, and R. M. Fujimoto, 406--413. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[41]

Rubinstein, R. Y. and B. Melamed. 1998. Modern simulation and modeling. 1st ed. New York: Wiley-Interscience.

[42]

Rubinstein, R. Y. and A. Shapiro. 1993. Discrete event systems. New York: John Wiley and Sons.

[43]

Sacks, J., W. J. Welch, T. J. Mitchell and H. P. Wynn. 1989. Design and analysis of computer experiments. Statistical Science 4(4): 409--423.

[44]

Samarasinghe, S. 2006. Neural networks for applied sciences and engineering. New York: CRC Press.

Digital Library

[45]

Santner, T. J., B. J. Williams and W. I. Notz. 2003. The design and analysis of computer experiments, new York: Springer.

[46]

Sasena, M. J., P. Papalambros and P. Goovaerts. 2002. Exploration of metamodeling sampling criteria for constrained global optimization. Engineering Optimization 34: 263--278.

[47]

Simpson, T. W., A. J. Booker, D. Ghosh, A. A. Giunta, P. N. Koch and R.-J. Yang. 2004. Approximation methods in multi-disciplinary analysis and optimization: a panel discussion. Structural and Multidisciplinary Optimization 27(5): 302--313.

[48]

Simpson, T. W., J. J. Korte, T. Mauery and F. Mistree. 2001. Kriging models for global approximation in simulation-based multidisciplinary design optimization. AIAA Journal 39: 2233--2241.

[49]

Wang, H. and B. W. Schmeiser. 2008. Discrete stochastic optimization using linear interpolation. In Proceedings of the 2008 Winter Simulation Conference, ed. S. J. Mason, R. R. Hill, L. Mönch, O. Rose, T. Jefferson, J. W. Fowler, 502--508. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[50]

Wang, L. 2005. A hybrid genetic algorithm--neural network strategy for simulation optimization. Applied Mathematics and Computation 170(2): 1329--1343.

[51]

Weiser, A. S. E. Zarantonello. 1988. A note on piecewise linear and multilinear table interpolation in many dimensions. Mathematics of Computation 50(181): 189--196.

[52]

Yang, F., J. Liu, M. Tongarlak, B. E. Ankenman and B. L. Nelson. 2007. Metamodeling for cycle time-throughput-product mix surfaces using progressive model fitting. In Proceedings of the 2007 Winter Simulation Conference, ed. S. G. Henderson, B. Biller, M.-H. Hsieh, J. Shortle, J. D. Tew, and R. R. Barton, 322--330. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[53]

Yesilyurt, S. and A. T. Patera. 1995. Surrogates for numerical simulations; optimization of eddy-promoter heat exchangers. Computer Methods in Applied Mechanics and Engineering 121: 231--257.

Cited By

Montevechi JAmaral JMiranda Rdos Santos CBrito FMachado M(2023)Ensemble-Based Infill Search Simulation Optimization FrameworkProceedings of the Winter Simulation Conference10.5555/3643142.3643397(3059-3070)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3643142.3643397
Yang XHu JHu JPeng Y(2020)Asynchronous value iteration for markov decision processes with continuous state spacesProceedings of the Winter Simulation Conference10.5555/3466184.3466513(2856-2866)Online publication date: 14-Dec-2020
https://dl.acm.org/doi/10.5555/3466184.3466513
Barton R(2020)TutorialProceedings of the Winter Simulation Conference10.5555/3466184.3466309(1102-1116)Online publication date: 14-Dec-2020
https://dl.acm.org/doi/10.5555/3466184.3466309
Show More Cited By

Simulation optimization using metamodels

Recommendations

Multiobjective simulation optimization using enhanced evolutionary algorithm approaches
Multi-objective reliability-based optimization with stochastic metamodels

This paper addresses continuous optimization problems with multiple objectives and parameter uncertainty defined by probability distributions. First, a reliability-based formulation is proposed, defining the nondeterministic Pareto set as the minimal ...
Multi-objective optimization using teaching-learning-based optimization algorithm

Two major goals in multi-objective optimization are to obtain a set of nondominated solutions as closely as possible to the true Pareto front (PF) and maintain a well-distributed solution set along the Pareto front. In this paper, we propose a teaching-...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WSC '09: Winter Simulation Conference

December 2009

3211 pages

ISBN:9781424457717

General Chair:
Ann Dunkin
Palo Alto Unified School District
,
Program Chair:
Ricki G. Ingalls
Oklahoma State University

Sponsors

IIE: Institute of Industrial Engineers
INFORMS-SIM: Institute for Operations Research and the Management Sciences: Simulation Society
SIGSIM: ACM Special Interest Group on Simulation and Modeling
SCS: Society for Modeling and Simulation International

Publisher

Winter Simulation Conference

Publication History

Published: 13 December 2009

Check for updates

Qualifiers

Research-article

Conference

WSC09

Sponsor:

IIE
INFORMS-SIM
SIGSIM
SCS

WSC09: Winter Simulation Conference

December 13 - 16, 2009

Texas, Austin

Acceptance Rates

WSC '09 Paper Acceptance Rate 137 of 256 submissions, 54%;

Overall Acceptance Rate 3,413 of 5,075 submissions, 67%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
231
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Montevechi JAmaral JMiranda Rdos Santos CBrito FMachado M(2023)Ensemble-Based Infill Search Simulation Optimization FrameworkProceedings of the Winter Simulation Conference10.5555/3643142.3643397(3059-3070)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3643142.3643397
Yang XHu JHu JPeng Y(2020)Asynchronous value iteration for markov decision processes with continuous state spacesProceedings of the Winter Simulation Conference10.5555/3466184.3466513(2856-2866)Online publication date: 14-Dec-2020
https://dl.acm.org/doi/10.5555/3466184.3466513
Barton R(2020)TutorialProceedings of the Winter Simulation Conference10.5555/3466184.3466309(1102-1116)Online publication date: 14-Dec-2020
https://dl.acm.org/doi/10.5555/3466184.3466309
Fatma NMohd SRamamohan VMustafee N(2020)Primary healthcare delivery network simulation using stochastic metamodelsProceedings of the Winter Simulation Conference10.5555/3466184.3466277(818-829)Online publication date: 14-Dec-2020
https://dl.acm.org/doi/10.5555/3466184.3466277
Wang WWan HChang KHuschka TChick S(2016)Randomized block coordinate descendant strong for large-scale stochastic optimizationProceedings of the 2016 Winter Simulation Conference10.5555/3042094.3042183(614-625)Online publication date: 11-Dec-2016
https://dl.acm.org/doi/10.5555/3042094.3042183
Wang BHu J(2015)On the monotonic performance of stochastic kriging predictorsProceedings of the 2015 Winter Simulation Conference10.5555/2888619.2889130(3825-3833)Online publication date: 6-Dec-2015
https://dl.acm.org/doi/10.5555/2888619.2889130
Jian NHenderson S(2015)An introduction to simulation optimizationProceedings of the 2015 Winter Simulation Conference10.5555/2888619.2888819(1780-1794)Online publication date: 6-Dec-2015
https://dl.acm.org/doi/10.5555/2888619.2888819
Barton R(2015)TutorialProceedings of the 2015 Winter Simulation Conference10.5555/2888619.2888818(1765-1779)Online publication date: 6-Dec-2015
https://dl.acm.org/doi/10.5555/2888619.2888818
Qu HFu M(2014)Gradient Extrapolated Stochastic KrigingACM Transactions on Modeling and Computer Simulation10.1145/265899524:4(1-25)Online publication date: 18-Nov-2014
https://dl.acm.org/doi/10.1145/2658995
Qu HFu MRose OUhrmacher A(2012)On direct gradient enhanced simulation metamodelsProceedings of the Winter Simulation Conference10.5555/2429759.2429815(1-12)Online publication date: 9-Dec-2012
https://dl.acm.org/doi/10.5555/2429759.2429815
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten