Judgement and ranking: living with hidden bias

The complexity and subjectivity of the judgement task conceals the existence of biases that undermines the quality of the process. This paper presents a weighted aggregation function that attempts to reduce the influence of biased judgements on the final score. We also discuss a set of desirable properties. The proposed weighted aggregation function is able to correct the ‘nationalism bias’ found by Emerson et al. (Am Stat 63(2):124–131, 2009) in the 2000 Olympic Games diving competition and suggest the possibility of a ‘reputation bias’. Our results can be applied to judgement sports and other activities that require the aggregation of several personal evaluations.