A methodology for sensitivity analysis of models fitted to data using statistical methods

A simple methodology is presented for sensitivity analysis of models that have been fitted to data by statistical methods. Such analysis is a decision support tool that can focus the effort of a modeller who wishes to further refine a model and/or to collect more data. A formula is given for the calculation of the proportional reduction in the variance of the model ‘output’ that would be achievable with perfect knowledge of a subset of the model parameters. This is a measure of the importance of the set of parameters, and is shown to be asymptotically equal to the squared correlation between the model output and its best predictor based on the omited parameters. The methodology is illustrated with three examples of OR problems, an age-based equipment replacement model, an ARIMA forecasting model and a cancer screening model. The sampling error of the calculated percentage of variance reduction is studied theoretically, and a simulation study is then used to exemplify the accuracy of the method as a function of sample size.