Distributional Input Uncertainty
Pages 2617 - 2628
Abstract
The vast majority of the simulation input uncertainty literature focuses on estimating target output quantities that are real-valued. However, outputs of simulation models are random and real-valued targets essentially serve only as summary statistics. In this paper, we study the input uncertainty problem from a distributional view, namely we construct simultaneous confidence bands for the entire output distribution function. Our approach utilizes a novel test statistic that consists of the supremum of the sum of a Brownian bridge and a mean-zero Gaussian process whose covariance function is characterized by the influence function of the true output distribution function, which generalizes the Kolmogorov-Smirnov statistic to account for input uncertainty. We demonstrate how subsampling helps estimate the covariance function of the Gaussian process, thereby leading to an implementable estimation of the quantile of the test statistic and a statistically valid confidence band. Finally, we present some supporting numerical experiments.
References
[1]
Banks, J., J. Carson, B. Nelson, and D. Nicol. 2014. Discrete Event System Simulation. Fifth ed. Harlow: Pearson.
[2]
Barton, R., H. Lam, and E. Song. 2022. "Input Uncertainty in Stochastic Simulation". To appear in the Palgrave Handbook of Operations Research, Chapter 17.
[3]
Barton, R. R. 2012. "Tutorial: Input Uncertainty in Output Analysis". In Proceedings of the 2012 Winter Simulation Conference, edited by C. Laroque, J. Himmelspach, R. Pasupathy, O. Rose, and A. Uhrmacher, 1--12. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.
[4]
Barton, R. R., B. L. Nelson, and W. Xie. 2014. "Quantifying Input Uncertainty via Simulation Confidence Intervals". INFORMS Journal on Computing 26(1):74--87.
[5]
Barton, R. R., and L. W. Schruben. 1993. "Uniform and Bootstrap Resampling of Empirical Distributions". In Proceedings of the 1993 Winter Simulation Conference, edited by G. W. Evans, M. Mollaghasemi, E. Russell, and W. Biles, 503--508. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.
[6]
Barton, R. R., and L. W. Schruben. 2001. "Resampling Methods for Input Modeling". In Proceedings of the 2001 Winter Simulation Conference, edited by B. A. Peters, J. S. Smith, D. J. Medeiros, and M. W. Rohrer, 372--378. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.
[7]
Cheng, R. C., and W. Holland. 1998. "Two-Point Methods for Assessing Variability in Simulation Output". Journal of Statistical Computation Simulation 60(3):183--205.
[8]
Cheng, R. C., and W. Holland. 2004. "Calculation of Confidence Intervals for Simulation Output". ACM Transactions on Modeling and Computer Simulation (TOMACS) 14(4):344--362.
[9]
Cheng, R. C., and W. Holloand. 1997. "Sensitivity of Computer Simulation Experiments to Errors in Input Data". Journal of Statistical Computation and Simulation 57(1--4):219--241.
[10]
Chick, S. E. 2001. "Input Distribution Selection for Simulation Experiments: Accounting for Input Uncertainty". Operations Research 49(5):744--758.
[11]
Corlu, C. G., A. Akcay, and W. Xie. 2020. "Stochastic Simulation Under Input Uncertainty: A Review". Operations Research Perspectives 7:100162.
[12]
Ghosh, S., and H. Lam. 2019. "Robust Analysis in Stochastic Simulation: Computation and Performance Guarantees". Operations Research 67(l):232--249.
[13]
Henderson, S. G. 2003. "Input Modeling: Input Model Uncertainty: Why Do We Care and What Should We Do About It?". In Proceedings of the 2003 Winter Simulation Conference, edited by S. Chick, P. J. Sanchez, D. Ferrin, and D. J. Morrice, 90--100. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.
[14]
Lam, H. 2016. "Advanced Tutorial: Input Uncertainty and Robust Analysis in Stochastic Simulation". In Proceedings of the 2016 Winter Simulation Conference, edited by T. M. K. Roeder, P. I. Frazier, R. Szechtman, E. Zhou, T. Huschka, and S. E. Chick, 178--192. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.
[15]
Lam, H., Z. Liu, and H. Qian. 2022. "Efficient Input Uncertainty Quantification Without Nested Simulation: A Subsampled Infinitesimal Jackknife Approach". Preprint.
[16]
Lam, H., and H. Qian. 2016. "The Empirical Likelihood Approach to Simulation Input Uncertainty". In Proceedings of the 2016 Winter Simulation Conference, edited by T. M. K. Roeder, P. I. Frazier, R. Szechtman, E. Zhou, T. Huschka, and S. E. Chick, 791--802. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.
[17]
Lam, H., and H. Qian. 2017. "Optimization-Based Quantification of Simulation Input Uncertainty via Empirical Likelihood". arXiv preprint arXiv:1707.05917. https://arxiv.org/abs/1707.05917, accessed 13th May 2022.
[18]
Lam, H., and H. Qian. 2019. "Random Perturbation and Bagging to Quantify Input Uncertainty". In Proceedings of the 2019 Winter Simulation Conference, edited by N. Mustafee, K.-H. G. Bae, S. Lazarova-Molnar, M. Rabe, and C. Szabo, 320--331. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.
[19]
Lam, H., and H. Qian. 2021. "Subsampling to Enhance Efficiency in Input Uncertainty Quantification". Operations Research.
[20]
Law, A. M., W. D. Kelton, and W. D. Kelton. 2007. Simulation Modeling and Analysis, Volume 3. New York: Mcgraw-hill.
[21]
Lin, Y., E. Song, and B. L. Nelson. 2015. "Single-Experiment Input Uncertainty". Journal of Simulation 9(3):249--259.
[22]
Nelson, B. 2013. Foundations and Methods of Stochastic Simulation: A First Course. New York: Springer Science & Business Media.
[23]
Serfling, R. J. 2009. Approximation Theorems of Mathematical Statistics, Volume 162. New York: John Wiley & Sons.
[24]
Song, E., and B. L. Nelson. 2015. "Quickly Assessing Contributions to Input Uncertainty". IIE Transactions 47(9):893--909.
[25]
Song, E., and B. L. Nelson. 2019. "Input-Output Uncertainty Comparisons for Discrete Optimization via Simulation". Operations Research 67(2):562--576.
[26]
Song, E., B. L. Nelson, and C. D. Pegden. 2014. "Advanced Tutorial: Input Uncertainty Quantification". In Proceedings of the 2014 Winter Simulation Conference, edited by A. Tolk, S. Diallo, I. Ryzhov, L. Yilmaz, S. Buckley, and J. Miller, 162--176. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers, Inc.
[27]
Van der Vaart, A. W. 2000. Asymptotic Statistics, Volume 3. Cambridge: Cambridge University Press.
[28]
Wasserman, L. 2006. All of Nonparametric Statistics. New York: Springer Science & Business Media.
[29]
Xie, W., B. L. Nelson, and R. R. Barton. 2014. "A Bayesian Framework for Quantifying Uncertainty in Stochastic Simulation". Operations Research 62(6):1439--1452.
[30]
Zouaoui, F., and J. R. Wilson. 2003. "Accounting for Parameter Uncertainty in Simulation Input Modeling". Iie Transactions 35(9):781--792.
[31]
Zouaoui, F., and J. R. Wilson. 2004. "Accounting for Input-Model and Input-Parameter Uncertainties in Simulation". IIE Transactions 36(11):1135--1151.
Recommendations
Input Uncertainty Quantification via Simulation Bootstrapping
WSC '23: Proceedings of the Winter Simulation ConferenceInput uncertainty, which refers to the output variability arising from statistical noise in specifying the input models, has been intensively studied recently. Ignoring input uncertainty often leads to poor estimates of system performance. In the non-...
Input uncertainty quantification: advanced tutorial
WSC '14: Proceedings of the 2014 Winter Simulation Conference"Input uncertainty" refers to the (often unmeasured) effect of not knowing the true, correct distributions of the basic stochastic processes that drive the simulation. These include, for instance, interarrival-time and service-time distributions in ...
Input Uncertainty Quantification for Quantiles
WSC '22: Proceedings of the Winter Simulation ConferenceInput models that drive stochastic simulations are often estimated from real-world samples of data. This leads to uncertainty in the input models that propagates through to the simulation outputs. Input uncertainty typically refers to the variance of ...
Comments
Information & Contributors
Information
Published In
December 2022
3536 pages
Sponsors
In-Cooperation
- IIE: Institute of Industrial Engineers
- INFORMS-SIM: Institute for Operations Research and the Management Sciences: Simulation Society
- SCS: Society for Computer Simulation
Publisher
IEEE Press
Publication History
Published: 02 March 2023
Check for updates
Qualifiers
- Research-article
Conference
WSC '22
Sponsor:
Acceptance Rates
Overall Acceptance Rate 3,413 of 5,075 submissions, 67%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 10Total Downloads
- Downloads (Last 12 months)3
- Downloads (Last 6 weeks)0
Reflects downloads up to 08 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in