Comparison of three multicriteria methods to predict known outcomes

Major approaches to selection decisions include multiattribute utility theory and outranking methods. One of the most frustrating aspects of research in the relative performance of these methods is that data where the final outcome is known are not available. In the US, a great deal of effort has been devoted to statistically recording detailed performance characteristics of major league professional baseball. Every year there have been two to four seasonal competitions, with known outcome in terms of the proportion of contests won. Successful teams often have diverse characteristics, emphasizing different characteristics. SMART, PROMETHEE, and a centroid method were applied to baseball data over the period 1901–1991. Baseball has undergone a series of changes in style over that period, and different physical and administrative characteristics. Therefore the data were divided into decades, with the first five years used as a training set, and the last five years used for data collection. Regression was used to develop the input for preference selection in each method. Single-attribute utilities for criteria performance were generated from the first five years of data from each set. Relative accuracy of multicriteria methods was compared over 114 competitive seasons for both selecting the winning team, as well as for rank-ordering all teams. All the methods have value in supporting human decision making. PROMETHEE II using Gaussian preference functions and SMART were found to be the most accurate. The centroid method and PROMETHEE II using ordinal data were found to involve little sacrifice in predictive accuracy.