Constructing More Powerful Exact Tests of Equivalence from Binary Matched Pairs

In randomized clinical trials, it is often necessary to demonstrate that a new medical treatment does not substantially differ from a standard reference treatment. Formal testing of such ‘equivalence hypotheses’ is typically done by combining two one-sided tests (TOST). A quite different strand of research has demonstrated that replacing nuisance parameters with a null estimate produces P-values that are close to exact (Lloyd 2008a) and that maximizing over the residual dependence on the nuisance parameter produces P-values that are exact and optimal within a class (Röhmel & Mansmann 1999; Lloyd 2008a). The three procedures – TOST, estimation and maximization of a nuisance parameter – can each be expressed as a transformation of an approximate P-value. In this paper, we point out that TOST-based P-values will generally be conservative, even if based on exact and optimal one-sided tests. This conservatism is avoided by applying the three transforms in a certain order – estimation followed by TOST followed by maximization. We compare this procedure with existing alternatives through a numerical study of binary matched pairs where the two treatments are compared by the difference of response rates. The resulting tests are uniformly more powerful than the considered competitors, although the difference in power can range from very small to moderate.