Combining judgments for better decisions: A study for investigating effective combining schemes

Facing decision-making tasks, managers frequently make judgments. However, since managers are human beings, the efficiency of their judgments is limited. Two major sources of inefficiency in their judgments have been recognized: one is systematic deviations from normatively preferred decisions, so called bias or incorrect intuition, and the other is inconsistency in their judgments, i.e. erratic decision making variance. Rather than bias, variance is really expensive or damaging. Thus, if the inconsistency in managers judgments is removed, performance could be by far improved by virtue of the reduced random variance. One of the approaches to improve managerial judgment is to simply bring managers together by effectively moderating the random variance due to inconsistency. Focusing on combining judgments, this paper addresses many relevant issues such as why combining and how to combine judgments, and suggests methods and models to effectively aggregate subjective judgments. The paper conducts an experiment to validate the effectiveness of combining judgments over individual judgments. Various combining schemes are also evaluated in terms of their predictive accuracy. Among them, mean bias based weighting scheme turns out the best. However, when available information is not enough to estimate the expertise of judges, simple and robust equal weighting might be more efficient and productive. This urges an imperative future research on the issue of ‘how many and which ones to combine from a large set of experts.’