Analyzing consistency of experts' judgments: Case of assessing forest biodiversity

Incorporation of biodiversity considerations into forest planning requires methods for the operationalization of biodiversity and for the quantification of expert judgments concerning the future implications of alternative forest management plans. Applying expert knowledge in numerical decision support most probably involves much uncertainty. To reduce biases, the use of several independent experts seems advisable. However, the experts may disagree, and it is not always clear whether this is due to genuine disagreements, or because of misunderstandings or other errors that may occur in the elicitation process. In this study, an approach is presented for analyzing the uncertainties in expert judgments elicited by pairwise comparisons, and for improving the consistency of judgments. Ecological expertise is applied in the estimation of an evaluation model which can be used in assessing alternative forest plans with respect to biodiversity in optimization calculations. The approach is based on the combined use of HERO optimization method developed for tactical forest planning, Delphi technique, and variance components modeling. The Delphi technique may be helpful in understanding better the causes of the possible disagreements. Variance components modeling is used to estimate the changes in the views of 11 experts concerning the different components of biodiversity, in three Delphi rounds. It turned out that the views converged to some extent, but in one case an increase in shared inconsistency among judges was also detected.