research-article

Designing simulation experiments

Author:

Russell R. BartonAuthors Info & Claims

WSC '13: Proceedings of the 2013 Winter Simulation Conference: Simulation: Making Decisions in a Complex World

Pages 342 - 353

Published: 08 December 2013 Publication History

Abstract

Much effort goes into building and validating simulation models. Similar care is necessary in the use of the model to support decision making. Good simulation experiment design is important to get useful and valid insights for specific management questions. This introductory tutorial gives an overview of experiment design techniques for justifying and planning a series of simulation runs to uncover the impact of system design parameters on simulation output performance. Graphical methods are emphasized.

References

[1]

Banks, J., J. S. Carson, B. L. Nelson, and D. M. Nicol. 2009. Discrete-Event System Simulation, 5^th ed. Upper Saddle River, New Jersey: Prentice-Hall.

[2]

Barton, R. R. 1997. Pre-experiment planning for designed experiments: graphical methods. Journal of Quality Technology 29, 307--316.

[3]

Barton, R. R. 1998. Design-plots for factorial and fractional-factorial designs. Journal of Quality Technology 30, 40--54.

[4]

Barton, R. R. 1999. Graphical Methods for the Design of Experiments. New York: Springer-Verlag.

[5]

Barton, R. R. 2010. Simulation experiment design. In Proceedings of the 2010 Winter Simulation Conference, ed. B. Johansson, S. Jain, J. Montoya-Torres, J. Hugan, and E. Yücesan, 75--86. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[6]

Beeler, M. F., D. M. Aleman and M. W. Carter. 2012. A large simulation experiment to test influenza pandemic behavior. In Proceedings of the 2012 Winter Simulation Conference, ed. C. Laroque, J. Himmelspach, R. Pasupathy, O. Rose, and A. M. Uhrmacher. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[7]

Kleijnen, J. P. C. 2008a. Design and Analysis of Simulation Experiments. New York: Springer.

Digital Library

[8]

Kleijnen, J. P. C. 2008b. Design of experiments: overview. In Proceedings of the 2008 Winter Simulation Conference, ed. S. J. Mason, R. R. Hill, L. Mönch, O. Rose, T. Jefferson, J. W. Fowler, 479--488. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[9]

Kleijnen, J. P. C. and R. G. Sargent. 2000. A methodology for fitting and validating metamodels in simulation. European Journal of Operational Research 120: 14--29.

[10]

Law, A. M. 2009. How to build valid and credible simulation models. In Proceedings of the 2009 Winter Simulation Conference, ed. M. D. Rossetti, R. R. Hill, B. Johansson, A. Dunkin and R. G. Ingalls, 24--33. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[11]

Law, A. M. 2007. Simulation Modeling and Analysis, 4th ed. New York: McGraw-Hill.

Digital Library

[12]

Lin, D. K. J. 1995. Generating systematic supersaturated designs. Technometrics 37: 213--225.

[13]

Montgomery, D. C. 2009. The Design and Analysis of Experiments, 7th ed. New York: John Wiley and Sons.

[14]

Saaty, T. L. 1980. The Analytic Hierarchy Process. New York: McGraw-Hill.

[15]

Sanchez, S. M. and H. Wan. 2012. Work smarter, not harder: a tutorial on designing and conducting simulation experiments. In Proceedings of the 2012 Winter Simulation Conference, ed. C. Laroque, J. Himmelspach, R. Pasupathy, O. Rose, and A. M. Uhrmacher. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

[16]

Sanchez, S. M., H. Wan and T. W. Lucas. 2009. Two-phase screening procedure for simulation experiments. ACM Transactions on Modeling and Computer Simulation 19: 1--24.

Digital Library

[17]

Sargent, R. G. 2011. Verification and validation of simulation models. In Proceedings of the 2011 Winter Simulation Conference, ed. S. Jain, R. R. Creasey, J. Himmelspach, K. P. White, and M. Fu, 183--198. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers.

Digital Library

Cited By

Barton R(2020)TutorialProceedings of the Winter Simulation Conference10.5555/3466184.3466309(1102-1116)Online publication date: 14-Dec-2020
https://dl.acm.org/doi/10.5555/3466184.3466309
Ryzhov IZhang QChen Y(2020)Advanced statistical methodsProceedings of the Winter Simulation Conference10.5555/3466184.3466186(1-15)Online publication date: 14-Dec-2020
https://dl.acm.org/doi/10.5555/3466184.3466186
Law A(2017)A tutorial on design of experiments for simulation modelingProceedings of the 2017 Winter Simulation Conference10.5555/3242181.3242221(1-15)Online publication date: 3-Dec-2017
https://dl.acm.org/doi/10.5555/3242181.3242221
Show More Cited By

Designing simulation experiments
1. Applied computing

Recommendations

Designing and deploying online field experiments
WWW '14: Proceedings of the 23rd international conference on World wide web

Online experiments are widely used to compare specific design alternatives, but they can also be used to produce generalizable knowledge and inform strategic decision making. Doing so often requires sophisticated experimental designs, iterative ...
Complex simulation experiments made easy
WSC '18: Proceedings of the 2018 Winter Simulation Conference

Diverse methods for complex simulation experiments can contribute to developing and gaining insights into simulation models, for example simulation-based optimization, sensitivity analysis, or statistical model-checking. An effective tool for conducting ...
Setting up simulation experiments with SESSL
WSC '12: Proceedings of the Winter Simulation Conference

Setting up simulation experiments is hard, even more so as simulation systems usually offer only custom interfaces for this task (e.g., a graphical user interface or a programming interface). This steepens the learning curve for experimenters, who have ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WSC '13: Proceedings of the 2013 Winter Simulation Conference: Simulation: Making Decisions in a Complex World

December 2013

4386 pages

ISBN:9781479920778

General Chair:
Raymond R. Hill
Air Force Institute of Technology
,
Program Chair:
Michael E. Kuhl
Rochester Institute of Technology

Sponsors

IIE: Institute of Industrial Engineers
INFORMS-SIM: Institute for Operations Research and the Management Sciences: Simulation Society
ASA: American Statistical Association
SIGSIM: ACM Special Interest Group on Simulation and Modeling
SCS: Society for Modeling and Simulation International
ASIM: Arbeitsgemeinschaft Simulation
IEEE/SMCS: Institute of Electrical and Electronics Engineers/Systems, Man, and Cybernetics Society
NIST: National Institute of Standards & Technology

Publisher

IEEE Press

Publication History

Published: 08 December 2013

Check for updates

Qualifiers

Research-article

Conference

WSC '13

Sponsor:

IIE
INFORMS-SIM
ASA
SIGSIM
SCS
ASIM
IEEE/SMCS
NIST

WSC '13: Winter Simulation Conference

December 8 - 11, 2013

D.C., Washington

Acceptance Rates

Overall Acceptance Rate 3,413 of 5,075 submissions, 67%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
94
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Barton R(2020)TutorialProceedings of the Winter Simulation Conference10.5555/3466184.3466309(1102-1116)Online publication date: 14-Dec-2020
https://dl.acm.org/doi/10.5555/3466184.3466309
Ryzhov IZhang QChen Y(2020)Advanced statistical methodsProceedings of the Winter Simulation Conference10.5555/3466184.3466186(1-15)Online publication date: 14-Dec-2020
https://dl.acm.org/doi/10.5555/3466184.3466186
Law A(2017)A tutorial on design of experiments for simulation modelingProceedings of the 2017 Winter Simulation Conference10.5555/3242181.3242221(1-15)Online publication date: 3-Dec-2017
https://dl.acm.org/doi/10.5555/3242181.3242221
Mertens KLorscheid IMeyer M(2015)Structural equation modeling for simulation metamodelingProceedings of the 2015 Winter Simulation Conference10.5555/2888619.2888698(701-712)Online publication date: 6-Dec-2015
https://dl.acm.org/doi/10.5555/2888619.2888698
Ozik JWilde MCollier NMacal C(2014)Adaptive Simulation with Repast Simphony and SwiftRevised Selected Papers, Part I, of the Euro-Par 2014 International Workshops on Parallel Processing - Volume 880510.1007/978-3-319-14325-5_36(418-429)Online publication date: 25-Aug-2014
https://dl.acm.org/doi/10.1007/978-3-319-14325-5_36

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten