Coherence and calibration in expert probability judgement

Many decision-aiding technologies require valid probability judgements to be elicited from domain experts. But how valid are experts’ probability judgements? It is of considerable practical importance to identify the conditions which affect the quality of these judgements. Descriptive psychological models permit the identfication of situations in which judgement is likely to be poor, and suggest methods by which judgement may be improved. The authors describe two approaches to the assessment of the quality of probability judgement-calibration and coherence-and review research findings following from these two approaches. This review is carried out within a framework of the psychological processes required to make a probability judgement. Three rival psychological models of probability judgement are located within this framework, and are evaluated in the light of the empirical findings. The authors conclude that none of these three models is unequivocally supported by the empirical data. They suggest that this may be the case because the models, experimental tasks and measurement techniques are not sophisticated enough. The authors make some specific proposals for the resolution of these problems.