Evaluating models for classifying customers in retail banking collections

When seeking to establish a repayment strategy with delinquent borrowers, it is useful to determine how they are likely to behave, so that an optimal use of resources can be made. We examine two behavioural classifications (‘settle immediately’ versus ‘not settle immediately’, and ‘make some repayment’ versus ‘make no repayment’) and apply a variety of rules for predicting into which class each customer is likely to belong. Since no such rule will yield perfect predictions, the way in which performance is evaluated is crucial in choosing a good rule, and hence subsequently in obtaining accurate predictions of likely future behaviour. We examine some popular standard performance evaluation criteria, showing that they have major weaknesses. We describe and illustrate the use of an alternative measure that overcomes these weaknesses.