Evaluating probabilities: Asymmetric scoring rules

Evaluating probabilities: Asymmetric scoring rules

0.00 Avg rating0 Votes
Article ID: iaor19951708
Country: United States
Volume: 40
Issue: 11
Start Page Number: 1395
End Page Number: 1405
Publication Date: Nov 1994
Journal: Management Science
Authors:
Keywords: measurement, probability, values, statistics: distributions, decision theory
Abstract:

Proper scoring rules are overall evaluation measures that reward accurate probabilities. Specific rules encountered in the literature and used in practice are invariably symmetric in the sense that the expected score for a perfectly-calibrated probability assessor (or model generating probabilities) is minimized at a probability of one-half. A family of asymmetric scoring rules that provide better measures of the degree of skill inherent in the probabilities and render scores that are more comparable in different situations is developed here. One member of this family, a quadratic asymmetric rule, is applied to evaluate an extensive set of precipitation probability forecasts from the U.S. National Weather Service. Connections to previous characterizations of proper scoring rules are investigated, and some relevant issues pertaining to the design of specific asymmetric rules for particular inferential and decision-making problems are discussed briefly.

Reviews

Required fields are marked *. Your email address will not be published.