Article ID: | iaor20117915 |
Volume: | 187 |
Issue: | 1 |
Start Page Number: | 159 |
End Page Number: | 183 |
Publication Date: | Jul 2011 |
Journal: | Annals of Operations Research |
Authors: | Hoshino R, Coughtrey D, Sivaraja S, Volnyansky I, Auer S, Trichtchenko A |
Keywords: | law & law enforcement, measurement |
Drummond and Holte introduced the theory of cost curves, a graphical technique for visualizing the performance of binary classifiers over the full range of possible class distributions and misclassification costs. In this paper, we use this concept to develop the Improvement Curve, a new performance metric for predictive models. Improvement curves are more user‐friendly than cost curves and enable direct inter‐classifier comparisons. We apply improvement curves to measure risk‐assessment processes at Canada’s marine ports. We illustrate how implementing even a basic predictive model would lead to improved efficiency for the Canada Border Services Agency, regardless of class distributions or misclassification costs.