Model Validation and Testing in Simulation: a Literature Review

Abstract

Model validation is a key activity undertaken during the model development process in simulation. There is a large body of literature on model validation, albeit there exists little convergence in terms of the definitions, types of validity, and tests used. Yet it is not clear what standards should be taken into consideration to avoid developing what could be considered to be invalid or wrong models. In this paper we examine existing literature on model validation with the view to identifying the existing validation approaches and types of tests used to assess model validity. In this review we focus our attention on three domains that usually overlap in methods and techniques: general Operational Research (OR), Modelling & Simulation (M&S) and Computer Science (CS). We analyze each field to identify the aspects of validity considered including the tests used, the validation approach taken, i.e. the suggested level of validity achieved (if this applies) and the reported outcome. The analysis shows that there are common validation practices used in all three fields as well as new ideas that could be adopted in discrete event simulation. Some main points of concurrence include the lack of universal validation, the continuous need for validation, and, the indispensable need for modelers and users to work closely together during the model validation process. This review provides an initial categorization of literature on model validation which can in turn be used as a basis for future work in investigating how and to what extent models are considered sufficiently valid.