Influence of user participation on decision support systems use and decision accuracy

The paper reports a study of the impact on user satisfaction and forecast accuracy of user involvement in the design of a forecasting decision support system (FDSS). Two versions of an FDSS were tested via a laboratory study. Version 1 allowed the user control over all aspects of the system including the ‘look’ of various screen elements and, most importantly, the model to be used could be selected (and tested) from a number of alternative forecasting models provided within the FDSS. In contrast, Version 2 did not allow the user to modify the ‘look’ of the screen, and also provided no opportunity for model selection: this feature was carried out optimally by the FDSS. The user was told the advantage of optimal model selection. Both versions finished by asking the user to either accept the forecast (displayed as a point on the time-series graph) or to modify it via the mouse if unhappy with it. Results showed a much greater satisfaction with the forecasts provided by Version 1, confirming the importance of user involvement. Users of Version 1, in about half the cases, selected poor models with high forecast error. Where a model close to optimal was selected, the accuracy of Version 1 users greatly outperformed low involvement Version 2 users. Overall, however, the accuracy of the final forecasts for users of Version 1 was slightly inferior to that of users of Version 2. Measurements of ease of use and usefulness showed no real differences between the two versions.