A comparison of two probability encoding methods: Fixed probability vs. Fixed variable values

We present the results of an experiment comparing two popular methods for encoding probability distributions of continuous variables in decision analysis: eliciting values of a variable, X, through comparisons with a fixed probability wheel and eliciting the percentiles of the cumulative distribution, F(X), through comparisons with fixed values of the variable. We show slight but consistent superiority for the fixed variable method along several dimensions such as monotonicity, accuracy, and precision of the estimated fractiles. The fixed variable elicitation method was also slightly faster and preferred by most participants. We discuss the reasons for its superiority and conclude with several recommendations for the practice of probability assessment.