Article ID: | iaor20051491 |
Country: | United Kingdom |
Volume: | 32 |
Issue: | 4 |
Start Page Number: | 323 |
End Page Number: | 332 |
Publication Date: | Aug 2004 |
Journal: | OMEGA |
Authors: | Phillips-Wren Gloria E., Hahn Eugene D., Forgionne Giusseppi A. |
Keywords: | artificial intelligence: decision support |
In the literature, decision support systems (DSSs) have typically been evaluated on only a single criterion such as the outcome from decision making. However, it is clear that DSSs simultaneously have a critical impact on the process-oriented aspects of decision making, suggesting that a combination of both outcome and process criteria are highly relevant for DSS evaluation. Indeed, process characteristics are particularly crucial for web-based and real-time DSSs because of their ability to delivery timely, current information through features such as just-in-time information, real-time processing, on-line transaction processing, connectivity and globally up-to-date information. In this underexplored area, we propose a framework to evaluate DSSs that combines outcome- and process-oriented evaluation measures. The approach is demonstrated in the context of a real-time threat criticality detection DSS. Investigations are conducted using a multicriteria decision-making method called the Analytic Hierarchy Process (AHP) as well as a newly developed stochastic enhancement of AHP. We find that the real-time DSS offered a significant improvement in terms of process-related characteristics. However, it did not offer a statistically significant improvement in terms of outcome-related characteristics. The importance of simultaneously addressing both sets of considerations is discussed.