Validation of simulation analysis methods for the Schruben-Margolin correlation-induction strategy

To analyze simulation experiments performed under the Schruben-Margolin strategy for assigning random number streams to individual runs, A. Nozari, S. Arnold, and C. Pegden developed special statistical methods for estimating a general linear metamodel (that is, a regression model) of a selected response variable expressed in terms of design variables (regressors) relevant to the target system. This paper describes a three-stage procedure for validating the use of these simulation analysis methods. Each stage of the validation procedure tests a key assumption about the behavior of the response across all points in the experimental design. The first stage checks for multivariate normality in the overall set of responses, the second stage checks for the Schruben-Margolin covariance structure among those responses, and the third stage checks for adequacy (goodness of fit) of the user-specified metamodel. To handle simulation experiments that display significant departures from the Schruben-Margolin covariance structure, the authors present alternative versions of the goodness-of-fit test and the follow-up analysis for the postulated metamodel that merely requires jointly normal responses. A numerical example illustrates the application of this validation procedure.