Comparison of weighting judgments in multiattribute utility measurement

This paper compares four weighting methods in multiattribute utility measurement: the ratio method, the swing weighting method, the tradeoff method and the pricing out method. 200 subjects used these methods to weight attributes for evaluating nuclear waste repository sites in the United States. The weighting methods were compared with respect to their internal consistency, convergent validity, and external validity. Internal consistency was measured by the degree to which ordinal and cardinal or ratio responses agreed with the same weighting method. Convergent validity was measured by the degree of agreement between the weights elicited with different methods. External validity was determined by the degree to which weights elicited in this experiment agreed with weights that were elicited with managers of the Department of Energy. In terms of internal consistency, the tradeoff method fared worst. In terms of convergent validity, the pricing out method turned out to be an outlier. In terms of external validity, the pricing out method showed the best results. While the ratio and swing methods are quite consistent and show a fair amount of convergent validity, their external validity problems cast doubt on their usefulness. The main recommendation for applications is to improve the internal consistency of the tradeoff method by careful interactive elicitation and to use it in conjunction with the pricing out method to enhance its external validity.