Variation of Preference Inconsistency When Applying Ratio and Interval Scale Pairwise Comparisons

Several studies on numerical rating in discrete choice problems address the tendency of inconsistencies in decision makers' measured preferences. This is partly due to true inconsistencies in preferences or the decision makers' uncertainty on what he or she really wants. This uncertainty may be reflected in the elicited preferences in different ways depending on the questions asked and methods used in deriving the preferences for alternatives. Some part of the inconsistency is due to only having a discrete set of possible judgments. This study examined the variation of preference inconsistency when applying different pairwise preference elicitation techniques in a five‐item discrete choice problem. The study data comprised preferences of five career alternatives elicited applying interval scale and numerically and verbally anchored ratio scale pairwise comparisons. Statistical regression technique was used to analyse the differences of inconsistencies between the tested methods. The resulting relative residual variances showed that the interval ratio scale comparison technique provided the greatest variation of inconsistencies between respondents, thus being the most sensitive to inconsistency in preferences. The numeric ratio scale comparison gave the most uniform preferences between the respondents. The verbal ratio scale comparison performed between the latter two when relative residual variances were considered. However, the verbal ratio scale comparison had weaker ability to differentiate the alternatives. The results indicated that the decision recommendation may not be sensitive to the selection between these preference elicitation methods in this kind of five‐item discrete choice problem. The numeric ratio scale comparison technique seemed to be the most suitable method to reveal the decision makers' true preferences. However, to confirm this result, more studying will be needed, with an attention paid to users' comprehension and learning in the course of the experiment.