Group preference aggregation rules: The results of a comparative analysis with in situ data

Committees are sometimes engaged in tasks concerned with establishing ranked priorities and rationing scarce resources. However there is no generally accepted approach to combining judgements from individual members of a committee to form a group consensus. There are issues concerned with the make-up of a committee, its size and history, and the procedures used to produce committee decisions. This article presents the results of an analysis of data from 28 separate committees engaged on an actual ranking task concerned with the selection of research proposals for public sector funding. This paper explores whether the ranked lists produced by the committees are influenced by the position of a particular proposal in the agenda, the financial value of the proposal, the scoring scale used by the committees, or an ‘out of line’ rating by a single committee member. The results suggest that the last two factors can have a significant effect on the ranked positions produced by the committees.