A multiple-criteria framework for evaluation of decision support systems

A multiple-criteria framework for evaluation of decision support systems

0.00 Avg rating0 Votes
Article ID: iaor20051491
Country: United Kingdom
Volume: 32
Issue: 4
Start Page Number: 323
End Page Number: 332
Publication Date: Aug 2004
Journal: OMEGA
Authors: , ,
Keywords: artificial intelligence: decision support
Abstract:

In the literature, decision support systems (DSSs) have typically been evaluated on only a single criterion such as the outcome from decision making. However, it is clear that DSSs simultaneously have a critical impact on the process-oriented aspects of decision making, suggesting that a combination of both outcome and process criteria are highly relevant for DSS evaluation. Indeed, process characteristics are particularly crucial for web-based and real-time DSSs because of their ability to delivery timely, current information through features such as just-in-time information, real-time processing, on-line transaction processing, connectivity and globally up-to-date information. In this underexplored area, we propose a framework to evaluate DSSs that combines outcome- and process-oriented evaluation measures. The approach is demonstrated in the context of a real-time threat criticality detection DSS. Investigations are conducted using a multicriteria decision-making method called the Analytic Hierarchy Process (AHP) as well as a newly developed stochastic enhancement of AHP. We find that the real-time DSS offered a significant improvement in terms of process-related characteristics. However, it did not offer a statistically significant improvement in terms of outcome-related characteristics. The importance of simultaneously addressing both sets of considerations is discussed.

Reviews

Required fields are marked *. Your email address will not be published.