The evaluation of research papers (or...how to get an academic committee to agree on something)

The College of Commerce and Finance at Villanova University sponsors an annual research awards program. Its objectives are to encourage and reward faculty research efforts. The process of selecting papers to win the award turned out to be far more complex than was initially expected. After many unsuccessful attempts at developing a manageable procedure, the analytic hierarchy process was used to structure the selection process. This approach has been used for the last three years without any serious problems. The faculty and external advisors using the system believe that it can better capture the subjective judgments of the evaluators than any of the previous approaches.