Turning datamining into a management science tool: New algorithms and empirical results

This article develops and illustrates a new knowledge discovery algorithm tailored to the action requirements of management science applications. The challenge is to develop tactical planning forecasts at the stock keeping unit level. We use a traditional market-response model to extract information from continuous variables and use datamining techniques on the residuals to extract information from the many-valued nominal variables, such as the manufacturer or merchandise category. This combination means that a more complete array of information can be used to develop tactical planning forecasts. The method is illustrated using records of the aggregate sales during promotion events conducted by a 95-store retail chain in a single trading area. In a longitudinal cross validation, the statistical forecast (Promocast™) predicted the exact number of cases of merchandise needed in 49% of the promotion events and was within ± one case in 82% of the events. The dataminer developed rules from an independent sample of 1.6 million observations and applied these rules to almost 460,000 promotion events in the validation process. The dataminer had sufficient confidence to make recommendations on 46% of these forecasts. In 66% of those recommendations, the dataminer indicated that the forecast should not be changed. In 96% of those promotion events where “no change” was recommended this was the correct “action” to take. Even including these “no change” recommendations, the dataminer decreased the case error by 9% across all promotion events in which rules applied.