The primary objective of this paper is to describe how site selection effects can influence the safety effectiveness of treatments. More specifically, the goal is to quantify the bias for the safety effectiveness of a treatment as a function of different entry criteria as well as other factors associated with crash data, and propose a new method to minimize this bias when a control group is not available. The study objective was accomplished using simulated data. The proposed method documented in this paper was compared to the four most common types of before–after studies: the Naïve, using a control group (CG), the empirical Bayes (EB) method based on the method of moment (EBMM), and the EB method based on a control group (EBCG). Five scenarios were examined: a direct comparison of the methods, different dispersion parameter values of the Negative Binomial model, different sample sizes, different values of the index of safety effectiveness (
), and different levels of uncertainty associated with the index. Based on the simulated scenarios (also supported theoretically), the study results showed that higher entry criteria, larger values of the safety effectiveness, and smaller dispersion parameter values will cause a larger selection bias. Furthermore, among all methods evaluated, the Naïve and the EBMM methods are both significantly affected by the selection bias. Using a control group, or the EBCG, can mutually eliminate the site selection bias, as long as the characteristics of the control group (truncated data for the CG method or the non‐truncated sample population for the EBCG method) are exactly the same as for the treatment group. In practice, finding datasets for the control group with the exact same characteristics as for the treatment group may not always be feasible. To overcome this problem, the method proposed in this study can be used to adjust the Naïve estimator of the index of safety effectiveness, even when the mean and dispersion parameter are not properly estimated.
