Evaluating multiple-attribute decision support systems

Differences in the perceived usefulness of five multiple-attribute decision support systems versus a basic spreadsheet are empirically assessed. Twenty-four college faculty members participated in a repeated measures experiment in which they used decision support software for personal computers to analyse semistructured multiple-attribute problems. The software packages employed were Criterium, Expert Choice, Logical Decision, VIMDA, VISA and the spreadsheet package Quattro Pro. A questionnaire was developed to assess user-friendliness of the software, confidence in the procedure implemented by the software, and users' confidence in their results when employing the decision aids. Time to reach a decision was also measured. Significant differences were found in the overall way users ranked these aids, in the perceived user-friendliness and the confidence in procedure among aids, and in the time it took to arrive at a result. Comments from users are also reported.