Determining the level of agreement of expert estimates

Determining the level of agreement of expert estimates

0.00 Avg rating0 Votes
Article ID: iaor200937860
Country: United Kingdom
Volume: 8
Issue: 56
Start Page Number: 586
End Page Number: 600
Publication Date: Apr 2007
Journal: International Journal of Management and Decision Making
Authors:
Abstract:

This paper considers the techniques of determining the concordance of expert estimates. Expert methods of evaluating particular criteria may be practically used, for instance, in multi‐criteria decision support systems, if judgements are in good agreement. This is expressed by the concordance coefficient calculated by using expert‐provided data. The concordance coefficient can be calculated only if the considered criteria are ranked. If other evaluation methods are used by experts, the data should be transformed to rankings before the calculation of the concordance coefficient is made. There is the dependence of the agreement of expert estimates on a particular method used. The highest degree of agreement has been obtained by using a direct ranking method. The effect of equally assessed criteria, that is, the tied ranks, on the concordance coefficient and thereby on the level of expert judgements agreement is usually insignificant and cannot change the results of rating.

Reviews

Required fields are marked *. Your email address will not be published.