Control-variate models of common random numbers for multiple comparisons with the best

Using common random numbers (CRN) in simulation experiment design is known to reduce the variance of estimators of differences in system performance. However, when more than two systems are compared, exact simultaneous statistical inference in conjunction with CRN is typically impossible. The authors introduce control-variate models of CRN that permit exact statistical inference, specifically multiple comparisons with the best. These models explain the effect of CRN via a linear regression of the simulation output on ‘control variates’ that are functions of the simulation inputs. They establish theoretically, and illustrate empirically, that the control-variate models lead to sharper statistical inference in the sense that the probability of detecting differences in systems’ performance is increased.